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Abstract: 

Abstract of JP2003348598 

<P>PROBLEM TO BE SOLVED: To provide a method and apparatus for 
memory efficient compressed domain video processing and for fast inverse 
motion compensation using factorization and integer approximation. 
<P>SOLUTION: A method for reducing memory requirements needed to 
decode a bit stream comprises: receiving a video bit stream; decoding the frame 
of the bit stream into a discrete cosine transform (DCT) domain representation; 
identifying non-zero coefficients of the DCT domain representation; assembling a 
hybrid data structure; and inserting the nonzero coefficients of the DCT domain 
representation into the hybrid data structure. A method for performing inverse 
motion compensation is provided. The method initiates with receiving a video bit 
stream, and then, a transform matrix type is identified. The transform matrix type 
is either a half pixel matrix or a full pixel matrix. If the transform matrix type is a 
half pixel matrix, then the method includes applying a factorization technique to 
decode the bit stream corresponding to the half pixel matrix. If the transform 
matrix type is a full pixel matrix, then the method includes applying an integer 
approximation technique to decode the bit stream corresponding to the full pixel 
matrix. <P>COPYRIGHT: (C)2004,JPO 
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■y ^ *«*fc't&»ieflU£ffi*affl-*-4fcfctt7 , n* p 
7A^^^TV^4, 

[ 0 0 1 9 ] § ^t»iJ<0SOtWt:*J^r. BSSSrlStt-r 
«§fL^»«[iI8Sf--yr*^4. d^m«lsIRr-y7- 

(i, b*T#T-*<37W-AfcB«Wt&ft*T-*<0 
b'-y FXb >J-A«r«ttKifcA<OIMMI«*^TV^ 
4. ««®S&r-yTfcli, r-^^b'-y FXFU-ASr 

59ft (Mitf, oa) «8*£fca- f HM-*fctf>*>0R 
ftcOBRfllJSMtffc/W 7 'J -y FH^WaJ/SSifiM 
«EIRt -y rtJS «$ixt v>4 . 

[0020] mvmmmiz&^x . t*f^f n-rsrit 

tt-f4. ;«t'f*f3-7'li. AoT<4b'y FXF 

u-A*»4,«Haeava&<^ F^r-^Sraai-f 4«t 

ai^^$^^ftr3-r (VLD) ^^T'^4. V 
LDijI(tLTV^4S*^'(t;7D>y^^*-t4. <r<7)£J; 



« 

1 
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R*^fc7*n v ? i: ilftLT ^£±SE<97*7 V+ttL 

x-^h. zco±j&ryy^\t. nm^m MUX. DC 

[00 2 1 3 **W)*<ott<OJB«lfetXCJMWi. JWB 

comm. mzmfXTjkitzffittcowmtmz. ot ic$ 

[0022] 

WKXJ££ T t>*«BB*IOf?* I. i i: 

[0023] zz?mm&mw.Mi±. mmx-tr 
ft s wttffimi ztihXoiz, t'f *a w ^ 7-54 

•J -y Fr-^fllSSttaFfftSfifcT'a 7 tcoWzftm®. 
ftaiM (DCT) ffiRttJ^<0*>1>U*»*^i: 

5 C i: h . OTt <k 0 BiWicSWJ-t* J: 5 1 . t\ 

[00243 @2ii, wtRfaigmmtzxi. mm 



r-^\ TOWtfa-*' (VLD) Xr-vl 2 

42fttfR**ft (DQ) X-f-^'l 2 6tiffilS$il^h'7 

hAbv-^tnctmt&mmzm^t-ti. mmmm. 

ti. (MC) Xf-y'l 3 4TttRW4fc*>K* 

7lx-Ays'7 7ri:tB?(fni.^ ; E'J (MEM) 13 0t 
. MC 1 2 8St^MEM 1 3 4 **X^f»**|flt 

7 4-fa*7?;|/-7V)&{c, ^yi^y^x-f^-y 

(RLD) ^f-y 1 3 2&Z/i£DCT (IDCT) Xf-y'l 3 
AtfgfiZtlh . lot. fS^fc£ft£7'0 7:?<7)F*|g& 

£. 7V-A|*I<7)#7D7^<7)#«DCT^L*^L 

^tv^mem 1 3 ocof-^mm^mm-i z t t«t ->t. 

^< * V EMU . fr^r B»Oi«*^ L K> * U teffl S: 5 
0%M^-ritA i T'l'§. APalcOM^Ji. iSfi^DCT 

mzx i 'J t -9 <o«#*«Mf Jtoo 

Sfr2rM/-F?T7-ri»L§v«Xd{— A$r^t 
[002 5 3 -eciT. Sat'L^t^^'J^cOVH^I 

wjt-o^xmtti. immmx\ ^mx^&tx^ 

hT-XhOtzMZ^ A'7*y yJYtA >-H.263(C*fe!lL 
T V> I f" 3 - y X'h h TELENOR O fx* r 3 - y Sr ffiffl 

Uc. =5rfc. WTti»BB'tl)IU609o4'fcfiH.263b'7 

t>*>. Motion Picture Expert Group (MPEG) 1/2/4. H. 

261* if, t"r*T-^Sr*tl)i:%'5rDCT<-^E« 

t" 7 b x r- y - at- h umt h z. t * j t-# h . 

■C<0SMs«*»l«:«rffiC-f4«tt3^ fSBft (DC 
T) ««^S<0fc*Ol«K8l6&fflWr/l<rf'jXAllft 

«^-ri.^^yEaiffitt. ffiKawcojoa^*^. ^ 
^mn^m-hztiMmztth. §^>t. xe 

^ t y . PSNRCO®^ f>fp«t 5 . 
[00263 
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^iffl^ft*t>tt<5 0 4 4<D§£(D8 x 8ro^P^ f k 1 4 Z 

*K 1 4 6ft>if>0 4oft>£9:/p?*/ 1 ', //, fl &lfifi 14 4 

( A x, A y) lC*oT*fflU 1^7 1/- ACD* T'Sbf h^$t5^oy 

^^rtciDSJRS^o (A*>o, Ay>o) ftttTfctem* 

»^p*/n©/<7>-^ (8-w,h) , (w, 8-h) &tf <8-w,8-h) 
1] [0027] 

fk - Z c n/ fc « (2) 
m 

£ (2) 1?, ffH«ai= 1 4. j = 1,211. ^ir*f LTiMHt&tf5'7 h&fE 

^H^-rSo tt^Ctffl> 1H (-fefa) 8 x 8t*>#*T?fjT?a!>S 0 

[»2] [0028] 

(3) 



0/ 



^=^2=^=1. .1. ( 4 ) 



KR3] 



[00 29] 



C 22 C 42 ~ A-w • W 



[»4] 



( ? ) 



(a 0) )03-348598 (P20 0 3~e98 

t *flfi-C*a i t BE* S r i: # T $ * . 

(7) ©fflaK (premultiply) % 5*&&K#i:* 

(poet multiply) t , 

1=1 

\Zt£t>. 

::-P, c„W DCTT*>4„ (8) (±. 9»aic»C6.ivfc«F 1 ',...,F; 

CB»iU, F,«rllHl-t*. fT5U Cfctt, jj? DCT, 

k «*iS)DCTSr-&tp¥-»«^R5>ir*)S. tot. (8) «, ^ 
itSRi.!}Sfc^«jg«:jB s/i^i/^Kfcsror.stTiciftw-rs dots® 



(PDA)*, *Offi^NyH^HIMH»TO6-C«i<ai 

iso 7 -f -v v x ff * t v > o Jt r 7° >j y 3 >-i6j trc 

**. H.263»fUi, It'7 hi— h(7)t'T:*f«W3 
0 h' >y h- X b>J - A ? «y ? XMfTlVd >J £fg 

gLTlf^ft^WfflST**. T/PrfUXAtt. %MfiF 

mum/wn. yyuyyxm 
flcftsjiflt'r* 1 6 <o3»^rffi4^-r>' a y t-v^c- i- 

[ 0 0 3 1 ] ffit' y h U- ht'1*#ft£ix£t'f *(±. 

-CBHHLTt^llltn-Cii. H.263«fgcOC:i: SrtgLT 
v > * *\ a LT V tfitf i:X^r t'f* 3 - f -y ? SfgT *> 
WIHfcftCtWa-ftii:*^**. ##£T'(c. H.26 

a, c^sts^+tcfflmp (gop) mm&v a 

*-4 y l- 7 7 b-i, (I7U-A) t-e-ntSK 
ftivf y?-yv-K (P7U-A) ^Jtfc'Jttf^&S 

B§^W5C** i 'jiiS7W—A^t're0^iXl>O 

frLKtfb. mzmfe/mzmm ( me/mo ui$igH&£ : 



!t^n**a#ftro-t xB#t^«t § . 1 7 

l\ H.263^;j^KW*#-XA£^i*5T\ *? 
■f-ft,7°n-fex^c^< 1 1 1 3 2@H— Elxyn-^ 
*«7?d/d -v 9 *A V V 7^n •</ ^ fc LTflF^-ftL^r 

C 0 0 3 2 ] H5li. H.263«fST^B*iOfi^J^S: 
l&HJ!LTU6«ll!l&l!T"J>£. i<«I^TUS J: ofc. 

H. 263fflfi{4»#fflfl|fc:*B«Siia&«fflLTV^. Z 

comm-ii. *wmm#o. 5^mm (049, <;7. 

5, 4.5>;) ^#tS»#^h/HCJ:->T^S<li. x 

yz-m. *¥*n»i«f»t» SB^wwftt. a^i*^ 
fiBA i5o-u asfifiB i5o-2. mmfmc 150-3. b 

SfiffiD 150-4(i. mmmQMz&frix^&coiztt 

L» BSGBe 152-U B*{4Sf 152-2. Wm&Sg 15 
2-3(4, #BSMSHt*ffcbUTV^. *¥U$wmmZe 
= ( A+B+l ) > ; > ; 1 1 &b L . Sil^Offl^ $: f = ( A+C+l ) > ; > ; 1 

ktbh+z b tfX'Z 6 . * J FSiIB7ilSic7)|iH$g=(A+B 
+C+0+2)>;>;2ta*)1- i fc S . 
[0033] H6ARV6Wi-tit-eft, <-Xy4 yQ 

T?-y<nmmy'u>v?zuftmz&KmLtzi><7)X'b 

I. mz. RLDl 3 2i:IDCTl 3 4**MC2 87 4-FA 
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iaijrrh*r^i*)aj®ss«oii:*«T*6. H6bo siac. Bismf^/ii, mwury 

BWTtt, ffiltffitt&tB&E ^'i-^SrMC 1 2 87-4- >f yjfH^V^ oj6»tf)jft"tia6AWSBHittt^f3 
ft (compositing ) , X^r—^J #7*0 -y ?<fc$r tf)Tli$r< . 183t£Ji . ?'J'V £ V/*WW>$ri3l*# 

»fB«^a'3* t Sar , C*S. L*»L**«fe, t'r*3- 2oc7)««raT'PSNRffllS<l* i S : 5rl.t'T^^«-ri.. 

r -y ^o&s^w^tf . £mmz.y?-ms.mm® i o o 3 a ] 

#»«&IC. (i) , (ii) . (iii) . (iv) . (v) i^A-fcftttfc. H 6 A 
©SMftfa-^f, IDCT ^0^13 411, A^>X<S8x8^D-^i3 
«»®«*^SM««IJHE*-T5. fe^-\r*A' (Y, Cr, Cb) ©■ 

3(ie*^!a»ae*»oiffce>a^ai5UTv^. B6A »a «) t. swsrii 

(-25SSrS256)roffiHlC!> >) v MsifZlxb* @ 6 B © DCT i » 

*7—'S\zi±&m<o9 y •> irv^sfrnfe^, »2©fflSAit, »f««^c* 

0B6A ©MC/B7?12 8ti, S« © » £ ^ * h^*5*B8Ufc MEM 

1 3 oa»?>©B3ifi£y H6A©«(ii) -e. t > ujB^snri/'>nif. 

(HP) 4tmi 6 0#>5&©B*li©¥:9*A: »K *©SBA«rftt>ifi^iE© 
SE»(-*.»5. H6B©£ (iv) T. ¥B* (HP) *8ffl 1 60#, DCT« 
»fc«S«rff«t-pT, *©tt#£j&t>J£^IE;Rtt:&©*£cK:*.tf>$,, ^iMK&S: 
^MfflKlJlO»L.3t«, fcp — 3©«8*ii5«4-*-«. @6A©« (iii) T. fnM\ 
^B5.M6 2 bT?(0ix52S5)©!5HK^ !> *y f >^ * *tfclI8itt*3Efc-f . 

@6B -e, rat* SftHj&tt©* y ytr^^ii, - m» 



[00 3 5] 3»#$r<->tf. MEM1 3 0(i, »£fflhl<9 

SIBfHH^a-^tt. 7P-AA>y7rfr\ 
(Y.Cr, Cb) ISSrfiWWifcftt, Aot<l,7l/-A 

4:2: Otfy/'J y^§^CIFt*r^ti±^2 0 
O^fDA'^f bc^t y#&ttfttf&£>&V\ MEM1 3 0 

[0036] H7J2. #&HH<0 1 Hftffl £ <t I . £ 
ff*fl&tf «^7 , o-feXB*e07*o -y ? £££31$ UT 
i»J7n7^0T*S. 7 0 J: 9 ±05^380 

JtSSWC. 171— AO+O 

7n -y 9X\&?7 U-J*cri'P<7)7 r n vtHzftLX t'rjf 

n-y? 1 7 2tt^8x8tfWC»*. zm&X\ ffi 
laWKT^ffilBXIilT^ 0 ti i . If flktfi 

RL = binary rrrrl 11111111111' 
ftTfi^l 2h'-y h (llllllllllir ) (i % 7*n >y ^ 1 



?vx\ &m%m.x'm*±%<t£ix^%> m^itcox 

Zmz~tZ> Ztlzi.*), yn 7 ?l:§f>(:3 h iz 

-th, 7~u -y ? 1 7 6 tiifflSftTv^ vr-vrf&t 
mmm^m®miiz\)a&mz\mzmh. 

7>VV?X$mm±, 2ffi$rtOg3g (element) . 

1 7 8T«*>-r. iit, ^wwo^ywyn 
[0037] i zmmiza^x. (so 

— oof -5 Lfc^DSS^WH^yvy^x^-c* 

y^«Rl7 8Rt^l8 0fe:a<HUT^t. L*»t^ir 
2 rxrygMti^h h . #2ffl$rtosm (7V, u 

K)l) *%<r>m®&\ 6h'7 h^ioTie^fS, 
[0 038] 

(9) 
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7D'y7 184(i. tlHLS ix£ 7n -y7 182*>£>& 
&ZiXtzi*<r>X'hh . /n 7 ;i84li, DCTH 

048*^204 7CD$lMX'foZ>Zb£i iB MbfrX'fo&o 
07O7*o-y7 1 86(i, 7'D'y7 1 84fcIDCT*gft£ 
ff&o £f£co7o 7 1 7 2 (7)H^IS7'n ? 7Tft * . 
gifl(?)4h'7h (rrrr') l±y y<7)ffi£5tit LT^ 

tofcxx7--rv-7-y7#5e«£ft*. \ 5<r>mmi 

omzmmt® (zero amplitude) <9flSft#«< 7>2r« 
fctfc Afc . XA^-7y-t >7RL= " FO ' ft 

[00 3 9] **yj5rg*fr«£>tfc*>fc. SC£g£ft 
ftfttf 7 7 -fe X-f £ fcftcof - 7 fltit * BH% L * I ttltf 
fcMrK KcDr-fffim. otO, TW. aSS'J* 

£B»fr*SllL JtVEmwm&t. it»fiSr(S<H 

1 *#gg t T § . SCS8Kiffl-> fc* ^yjE 

mzmmt&tf. ftfe<DT-mmz£~>x&3r>cDtt 

r w < -y 7 tz> tzvwmktfmx-h h . m 2 (c . ay 

(sort and merge) Jftffr^'lTCftl>. 

[0 0 4 0] H8«i, 7yi^y^*3K^#8x8yp 

■y^oRBMMtfl.ai'^ftt:, JM*>>f yr y^^Srffi 

^T«8x8/D7?l 9 2-1*^1 9 2-4C07 



ODCT7o -y ^ tr^-fe^-f &ttt-teD|»&aB*»< 
[004 1 ] 09ARtf9Ni» 7W^-7<9T-7fi§ 

y t. h r- fimm*ti?tazm lx , ^a^as 

1^*. 09 AT, 7M^-7.c7)T-7fliji{2:, ttlfrf 
l>7M 4 y-f<y ^Xtfctt^tt^JlUtU^fc L«r 
K Li'LWf,, 7M<-7<7)T-7$itfc:{i**: 
'JBBBWfe&fcSru. 09BT\ yxh- (fcL<«. ^7 
Y IV ) r - 9 Wt tt $ 6 CMSVMff^t' 
ft*, -f f#£07/P:/yXA«j:t$A&tfB§fc 

9ffimzb^X%\%&<nWkfrWm\z%mztch* «fc 

ftijiarxui^w-tsii:^**. (0,20) + 

(0.620)= (0,640). fflM-?tc4 >T~/?Xtf : fmX'%,ft 
LTV*i*£fc:ii, DCTffiftj&WASii*. Witf. (0, 
-3)£t?A. DCrttflJnJCfcJ: 9 0C$r*>»£fctt, Da 

m.&mzti&. mm. <i, i3> + (4,-i3)=a,o). 

[0042] ^Ki, V^V^f-^flfiB^^yff 
Of- 7 ffijfti , ^aiOS^f £<MRr 6 4 coimJUaiM: 
£DCT{»>'&jf OftWC 1 hT1i*< 2A'-f h$- 

•TS>Ii:{iT'#^V\ JtftUXhXIi^fBi (SC) *S 
^7 h7Hi, ffiS«07U-f tit^ ±P8T-2. 5fgcO 
^ ^ y ffifil*^IfgT'ft h. LfrLtcifih. <7b /KOff 

Hi***. Wc, ^7 ^;^coE^4'0^fA&y : HlJ^iii^ 

ft* . 
[*n 







»^ 


K 






152 










304 










60 






2.5:1 




60+^— /< — 
K 






2.5:1 /<-^v 


sc <^/^^y 

v K 


70 






2.2:1 



[0043] SCiiSOVW 7' 'J -y Kr-7ffi3tt<t 0 . 

« i tm& lx nmffi^ttcsrA? ^7 sr-r * d 

fcA^CSS. @19Ac07^-fffijiOffilr»Sffla^S9B 
^ 7 - y -y Hr-7«j&*^yl$il^. A>f 7'J >y Ff- 



7'7U-f J±, 7o -y 7«tS LfefiS»«0Dafi6ft 
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£fmt&£olZHW&t>Zb&?£&. ZZX\ DCT 
fla&tf>R!i6 4*}jrcfc4. m34Tfc*<. HSitiM 
X7M09e*<*fti:, *ti£JtMl/0<%!J£Ea)l 
36**4. lflttWT. BKlMX7Htt. yoy^i, 
fc 0 8o<7)DCTfi&fSc£ £874 <k 3 CflWtS*iTV»6 . 

[ o o 4 4 ] 01 ott. *mm 1 USt^tJ: 4 , 

h/HH»Httr>W 7 0 v Y?-9W&0MllBtVh 
4. DCT7*n>y7 2 0 0-1, 200-2. 200-n 

DCT7'n-y7 20 0-ia>£>2 00-n{4. 02fcfcW 

7n«y 7 2 0 0 - 1 frk 2 0 0- nli„ t'r:*T-7<7) 
7U—A<D7*n-y7. fl^Jf, 07<?)7*n-y71 84. 
fcMjiftft^S. 707^200-1*^200-11 
<7)#7*o v 9<r8&MtiffiSAZti. XT W 

2 0 27*-^flBft«0tt>C#AS<l*. HJS1MX7W 
2 0 2 fcti. SS^-f X7o -y 7 2 0 4 - 1 frh 2 0 4 
- tiifibh . 1 UttsfllT*, #7'o -y 7 2 0 4-1 *» 2 

0 4-nJi, 8x8r-*«e<a«frC8o*Mcrfla&:£ 
ftf*T'#4*#£fc&-?T^4. 8-9 
ODCT$gfc£«874fl§j£(;:&o*:7Q -y 7fcH£§ix 

4 totii'5r< . mitzQMco^-A xzmm&z ttf 

T* 4 . jfcfca^fcJ: 3 K . 7n >y ?<0S»#Jt*T4 
k, **ifc^LT**yff«»4ftT*4. 
[004 5] 3I£*5?#. 01 OfcfcWC. DCT7'o y7 
200-1 H200-n« k'flXc: 8 ISriSI*. 4 
$&#Ao-a>4i§£. ^fflKt'f XXu y 7 2 
04-l*^204-n <0§M£j@;c 4 #«DCT$iM4 
tfw\*-70-^7 WU2 0 6^4ncA:fl£;ft.4. 

a*-7o~*7 h;i/2 0 6(i. «r^H X^-A'-7a 
-<7 ViVb LTttJS&h."0>4. *- a*-7 

D-<7 WWifSWT**4 . PUtf , 7'o 77200- 
lii, 9o<^N*DCTffi»Al~A9*^-CV^4. 
T\ DCTflaftA 1 -A8ttfflje^>f XXa -y 7 2 0 4 - 1 
Kntr-SfU)* 4 . DCT$j&A 9 l^-A'-7D-'< 7 h 
/P2 0 6(33t-$/h.4. 7*Oy7200-2«4, 10 
fflfllMlDCTffiSB 1 ~B 1 Oi-kA,X'^&. ZZX\ DCT 
{SffcB 1 ~B8(iH^^X7*n y7 2 0 4 - 2K3 f- 
S^tf, DCT$&B9&tfBl 014, 7V-^c07'D-y 
7ff£, ^--A'-7D-^7 WU206, 3 
t-£ft4. 4 yf7 7X^2 08 K(4, :*-A*-7a 
h/P2 0 6Oxyh-»J{cML-C. ttffrf 4I15&7 
-f XXo «y 7 2 0 4 - 1 frt> 2 0 4 - n £t8Si]74xy 
h'J* { A-3t^4. #xyh>J{iiy\M r-£**>£>, A > 
T-v7X&m4X\iimthZbifiX'$h. flSoT, 
DCT7*u -y 7200-1*^ 200-n iZttJ&f 4r~ 
9<T>7V-J±XM. A*-V2\ 0££*aJ7fctf>fc, 
m&HXTU4 2 0 20*-A-7D^ 1-/1-2 
0 6*>&<Or-?# { fflA£;fo$;h.4. ffiS^tyoB 



*WCS> 4 . 7&;b*> . DCT 7"n y 7 2 0 0 - 1 2 
0 0-nli. iikA,t!<Wte. 6 4«9*»tf#*fl9& 
4 XXo -y 7 2 0 4 - 1 ri><i 2 0 4 - nfc 
«»3 flfc 8 fflaT^MWHM -CiS^-f4 . So*T' 

$fi5ft#8S:j@*4k:*WN*-70-'<.7 p./U2 0 

[OO46]011A~11 Ci4, #*?U0) 1 UttHC J: 
4 , /W X U >y Hr~ 7fltjtO®^^ X7VA (rfflfc 

fetmizwmztimTZWLWtx^&ryyxfc 

4. HI 1 AT'. 2r>a)mmt£Z\Yis-yvx<m%.7" 

0 -y 7 S>£ 0 <0**DCT«M»OT*aR36< . & 2 2 0 SIX 
2 2 2 TfltaWlT ^4 . 7*a -y 7 0 ^Mr«MKtf)ft 
Ji 3 7 OJEfflt'fc 4 . -r^±>*>. 6 4fflc0^c03 
*>,WLt2H7 «<0^BKf ft*^MMS»T & 4 . 
Ill lB{i. SS^XTW-f^3ljD-r4k. jJw\- 
7o-K7 h/^-t'f X*^-t40T'. ^7 h/l/^tf 

XRVwmnmzmmzt 4 i t *«t*# 4 £ t & , 11 

1 lwmmvzfttLxmux. frntx^i. zz 

X\ i!2 2 0-l{i:iai \k<7M2 2Q<r>ZYfi/-'rvx 
izjcHtHX^h^izML. n.22 2- Htm 1 1AC0IR 

2 2 20CIFV- 7->-Xt*fi£LT^4. a««*4> 

JEffi{±fiT-t4^k* { ^4„ 01 1C{±. T 

VAO)%^m- (load factor) t^LT. TU-^tt 

«t^s<?)**"C*4ii:^Lrv^4. lsestw-c 

XTl/A£MA,i£. ZZX'i>. W.2 2 0-2(40 1 1 A 
<7)tg2 2 0<7)ClFv-7-yXfc*tfEU H2 2 2-214 
01 1A<0^S2 2 2<0CIF^-7->-X.fc*tJS5tTV^4, C 
cVmRi>\ ^-a'-7 a-<7 b n^W>f Xlz&Hz L 
rDaffi»*»2 0 0t:»i. A^H : ?Sr^9%k^l 
5%^H(Cffl$-t4. im&Khll. ffi&MXTV4 
{47'o 7 7 biz <0 8ffiRCIB£$ix4 t>^TJ4^< . 7* 
o 7 7 9 cof*gtco|S:(4ji U/Stt«ogtS:S^'V > 
ZttfWt>frX'b&. Sfet, ffl&MXT\'A<nm* 
cOXa 7 7 (£> LfeffiEOHjftC-r 4 d k 4 . « 
£tf, 8oO^SI:ft^-r4tg*Sr#t4XP<y72:, 
^U(f, 8xlc7)7*D77, 4x2^7*D77kLTi£ 
mtZZttfX'Zl-1i. 9oo«HS*«^4fl»3* 
Wf47'n772r, WiBf, 9xlc7)7*n77, 3x3 
«Xo 7 7 k LTi2?iJ-t 4 ^ k ^'T'# 4 . 
C 0 0 4 7 ] 0 1 2<4, 1 9SHM(c J: 4 . h'7 

hT, h v-&zm j 5it-t&cot,zgi-z>* : z ] jm^M.$:m 

4. ^C0*ffi(4. t'T*V-ybXbiJ-J>Z%(,m&* 
^U-^3y2 30*>^«r^4. 158«WC, fyhx 
h 'J -AliffiU- h h'7 hX h y -AT'*>4 . Witf . 
h*r'5)-7,hy-A$:. H.263. Motion Pictures Expert 



(&4) )03-348598 (P2003-598 

Group (MPEG-1/2/4) . H.26L Joint Photographic Exp 
ert Group (JPEG) ^rf, t*T*tt*H\3mttmm 
ZZttfTZZ. Z (Dl?&imz . ^l/-y 3 y23 

7l^-Afc|ijittft^ftl>T-:??)#Xu 7 9V>mt&0 

*m>%® (dct) fii«^ca^t$ni». i^-e. 

br^Ji. H2, 6B. 1 5K^L£T3-X&fc\ X* 
T*T-?ii. EIS$il£b>bXMJ-A£DCTiI«{ 

ffifBft«7*-^7r-£&-?-<:^l>. 7 l^-Ali-JgC: 
-^3^234 TDa«Wj«9l^MMHR 

t mmm ^tii6 4 <ma®m.n ? *> , 6 4 «t>*>jtK 
[0048] 81^^. Hi 2(cfc^T, .ro;frft«& 

A>f X'J 7 ^r-^m^T^yfiV-fh^V- 

y 3 V23 6 tottf,, A-fX'J 7 r-*T-*flfiU4» H5g 

•frX/CUS. «5WWr— 3*vvf X'J 7 Hr-^flBt 
(4. m 1 Ofc^LfcaROffljglM XXo 7 ^at^ix 
^4X:+-A-:7n-^7 r-;l/£-£tfHJglMX7W 
"C*S. itfWMSliiKfc:. ^lx-i/ 3 X2 3 8 til 

z zx-mmmmimmimtf^ xy •? kx- 
(c . b'r *r - * xn 7 ? (mammmvimim 

14. @£X4XTMl%^g£X4XXn7 7iWjitt 
2M»«<7)&*>\ t'r*T-:?Xn7?hlW 
«tt(t4>ftSli£X4XX0 7 70SM£jSI*.l>i:, Bt 





4MB$N1 (ms) 


b v Mfe 




772 


144 




243 


56336 




57 


39830 


jgDCT 


3 


42253 



[0051] ^3J5. %m\L $ fro ^ v igmrnm . ■& fn-ms. 

3b*f^T3-^^-f $yyyuy r 4)i>x'bh. - [*3] 



mm 


m&m (ms) 


b y 




9194 


56336 




1547 


144 




32 


39830 


JSDCT 


652 


340197 



ft 3 tih . 1 HJtBfl-e , 4 yf 7 7 XlHi . ' <- 7 

^#il5£lM XXo 7 7 fct. /vfX'J 

7 Fr-*ffi3ifc#^fii&0#ftfc: J: v\ *t 'J 3fM 

* He* * y RfSJI^ 5 0 %®$Tl> i 4: #T'£ h . 
[0049] &fc. f-^7^-AtMiSWt^ixl.# 
DCT««aW^*fl«W4A-f X'J 7 Kr-^fllJt(c« 

^<oa««^S(ca#fl:snT ^Bf. A>f x y 7 

tit. mmzmmi. %$mmmmmmziimtmA 

[0050] SIS|H.263b*TJrr3-X<0±^fi!«g* 

SVXXo7r-f 7SrfflV>T, 1.1 GHz^y 
f -f T A ( Pentium ; SflfSifl ) 4 Xob yfT^ff L 
T ^ h TEtEN0Rc7)H. 263 t*ftf"3 - XOttBg ^ 5 
4 yr-*-CH£tS. ^-AyJyT-fZU^k 
t % 'sAT&WSlbL&imLT . Xn7r^5{4. 1 

4 4 *>7 ^-A^rfi^-^'ti.^tst-i.imw^ra^ 

*2I4, SIIIAW. 263b'r^T3-X(7)^^ 

[^2] 



(a 5) 103-348598 (P2003 -§H, 9 8 



7 h—JxCDt! 7-B£YWfr&RGB(£3g«-f 

fltliWtaiM * * t U t> L < ti 7 U - A a* ? 7 r rt 



U-A*a#flrf4tf>fci»13. 6 7fM»*»-3THft. 
[0053] 



8 * ( 8 ) -e H W L X 0 (c , E 8 ff tt T © &± B S? <0 * ft # « « K: li 4 o ( TMd 
^■»ttn)ii\ttt^ti^. ra«f ti> 8 x 8«5fT?tJ:/n5'*/ : ",k:j£&*f5<l 



[»6] 

F t =7M } +7M 2 +lM i +TM 4 



[0054] 



(10) 



7M,=C (1 F',C„ (11) 

*4«. 5££BSS3E»fT?!l«r£ffiLTt>.5. rciT. c lV f± 8 x 8 © DCT ff W 



[H4] 



[0055] 8 x 8'mi<V?fcmzl±&* s 5 1 2c7)J»Jt 



7.*- At. 02. 6B. 1 St^U^-fT*^ Vfc 
fc^JtKSr^UT^S. #t-^71x-A*J3 5 2*«0 
SM»Bl7 *- V y hfcV^^h'r*7^— •?>•/ hSr-t* 1 

8>1 5-2 5 71/-A (fps) fc^dU- hVm^it-t 
[*5] 







71s- 


FPS 


<§# 




9.79 


144 


14.71 


sj-y ^/^TRI.HNORH.263 




14.17 


144 


10.16 


TMtf>5fe£ 8x8 ?T#i$f* 




9.82 


144 


14.66 






12.95 


144 


11.12 






14.85 


144 


9.70 


ft fcV* 




9.83 


144 


1465 


asm, PSNR> 



[0056] £EHHtf*awv r^-f >oxy 
^S5t (10) ttnH»*»£*4>'f ifc-C**. 7*o 

014 4 7U-- Afcf^HbLT. VvvVTvltVY 
W4/n 9 ?»3 6. 7%TM^1-&. 013l£, fir 

IX^&WmX'&Z. 7u»/7Ty4 *>Y24 0 
((w=8, h=4)) , 7'0'y7T9^^>'h24 2 ( (w= 
4. h=8)) . RV7a y?T74*>h244 ((w=8, 
h=8)) <0#*--^*qf*»<iT^&. £flfc*>W2 4 



0, 24 2. 244T't±g-*. Ml&t&7'v<y7t<?>* 

1. oa^tfj (smm) x\ yv v7T7j*>h 
coao. (w=8, \\=&)<7)mx\ *w&M$tz\&. 

1 2&V1 3<,zif;2tlX^&£oC 4r><nmm%w 



(€16) )03-348598 ( P 2 0 0 3 -ig"7f-tfi 



™» + 7M„„ + TM hf% + 1M hfi 



(12) 



hpfl J 



hpi2 



(13) 



[SS6] 









*¥SiBC*(Sl«ISl 



































[0057] ^mzTy^^yh^tltzftl-ypTt, 

cr^mmm^mmx'hh . ^mmmmiz x o , 1 m 

[0058] 02 <r>T3-y<?>ffi&y'n < y tn&tmt 
tz i *) EBKHWtt W<4 7*54 >-<7)^ax t - K s-± 

trt^tmi,. «2SU f 3T\ 52DCT70 -y 7 (7M 

mmii. mm® (632 ») tit^. 

sWiSjWCjSlr* (3 ms) . gJSMWZ\ MDCTii. 7 -f 
- F/\' y ^;U-Tc7)fit^ >hy7n -y 7&t>'f&Hf£gt 

[ 0 0 5 9 ] m 1 5li\ *»BJo l Stttflfc J; & . t'r 

ttfiT* n -y 7 gRig LT v >4 W^EPC* * . 



yA y\ti TfftX'Wt bti& . mnvyxrv ? m4vld 

1 24&tfDQl 26^kiZ^ (i) 2 5 2?S**. ± 
8Sc^7*5yf-T% AM 7*54 Vfct* ^'JffiHSl 2 8cr>|*| 

RLD&yiDCTS-B?*^i)tl.. 2*B<Z>X7Vvh 
14, »S=*Sm (MC) B#IC£ (ii) 2 5 4"CfB^-tS. » 

swjrat:. ( 7 ) let* k . ^ffl«jai^* s ± 

)£SfXl>*r&&f4a 5 <5£> . T 4 X7*W 1 3 6 fcfUjcTS 
fctf) t=j& ( i i i ) 2 5 6 T"3l7*Q Bfiflg-f S * 

DCTfl«&*tfH- &/S#>tDCT7'n 7 ?250* ! 7^- H 
A-y^/V-TCifAShl.. £ 0) 2 5 2TORLD1 3 
2 1 IDCT 1 3 4 fc<7)«££Ata£ ( ii ) 2 5 4T'?)DCT 
it, mi<mM?yA yOft^Xf— v"C^IDCT7'u-y 

T^4ffitfD«jBftx*-A(cJniTffl-&*?rffi i arHl 5 
IC^UcM^^tcJ: 0 2 0%<O^e-HT -yTSrHft. 

[*7] 



^16 


7*u y?<D% 




13 15 <D&(i)X'<D 
IDCT 


15% 




IS 15 </}£,(ii)t-<7> 
DCT 


63% 


y^(lDCT^S5»tn^^. T 

dct ^THii:^^ 


H 2 CO IDCT 


100% 


dct Kt^p y ^ f-Jiffl 



[0060] 1 HSfe0yT\ (lis 13) cojfefc 



tcH»^M»$<x6. ^sst (i7) <mmu. wmn 

^(Ai.A 2 ,A8.A,.^,A6)»t^WfttT5«J(D,H)^tr. i« 



(a 7) )03-348598 (P200 3~*gz98 



(16) (Dm^&zm&ftMZiltt&iRZmiii [»8 ] 

-t£i#T#S. -lixfi. JrSsUl 1. 13) iOt 

S = DA i A 2 A 3 MA 4 A i A i 



(14) 



(15) 



TM i ={DA,A 1 A,MA A A i A t )c n {DA,A 1 A^A 4 A i A i )'F' i 



(16) 



D = 



S3 



i i 
1 -1 



i 1 
1 



A 2 = 



1 1 

t 1 

1 -1 
-1 1 



1 I 
t 1 

I -1 
I .1 



•1-1 



I 1 

1 1 
1 



A 3 = 



I 



1 1 
4 1 



A 6 = 



1 1 
1 

«1 1 



I 1 

rt 0 



1 -1 
1 *l 



.1 



(17) 



D = diag{0.3536,0.2549 ) 0.2706,0.3007,0.3536,0.4500,0.6533 1 1.2814} (18) 



A = 0.7071,5=0.9239, C=0.3827 

[o o 6 1 1 ftot. 'mmmi. wwim-m^wn 

t»tih. fflU #8* (16) fc^tJ: 3C, ilTMi.cT) 

■A = ^iGJ = c n G\ , J w = G,c, 5 



(19) 

<rm ?)V-7ttjfi'ttbixh . fir?iJS (= G 0 G,)(i2OO 
JJL G 0 =DA,A 2 ^ (JiJFlJfc^ffWfi^) t. Gi=MA,A 5 A 

(24) TaaB*«fli{cHft4wws#ifc**0jfi-r4fc 

2) O+KftASfl* . 
[S9] 

= G lCj2 (20) 



c,,GJ = c 41 G' , AT W = = G,^ 

[Ski o] 

C hf\\ G, = ^JlG, , •/„ = G,C^,| 2 = G|C^,j2 



(21) 



(22) 



(23) 



F, = S[J h G' 0 F , x G (l J w + J h G' 0 F' l G <) K w 



(24) 



+ K„G' 0 F'fi s J w + KjP'F' fiJCjS 1 



(S. 8) )03-348598 (P200 3-*£e98 



[ftl 1] 



I -I —a 0 b a c 0 
1 1 -a -1 A 0 c 0 
1 1 -o -1 -b 0 -e 0 



1 -1 -a 0 
1 -1 a 0 



1 1 

0 0 
0 0 



a 1 
0 0 
0 0 



-b -a -c 0 
c -a -b 0 

c 0 -b -1 



0 0 
0 0 



0 
0 



0 
0 



[0 0 6 3] :;tli a=0.7071, b=0.9239, c=0.3827 
ti~&. u= V---,u 8 }.&lA'={v 1 ,"-,v 8 }i;{££-t6 

UK12] 



v,+v 2 



^ 2 = 



v,-v 2 



y 6 =y s -y< 



yi=y%-y* 



(25) 



(26) 



(27) 



(28) 



(29) 



(30) 



(31) 



y*=y*+y* 



(32) 



_y 9 =(*+c)(v 5 + v 7 ) 



(33) 



Jio = ^5 



(34) 



(35) 



3'|2=>9->'10->11 



(36) 



y n =yit-yn 



(37) 



«l =^2-77+^2 



(38) 



(39) 



"3=^6-J'.2 



(40) 



«4=>'2->',->'l 



12 



(41) 



«« = y\ +y 3 +M 4 +^13 -»i (43) 



(42) 



M 7 = 0 



(44) 



[0064] 



(45) 



(SL9) )03-348598 (P2003-ch"98 



If o 

X'IZ. 1 0 4®<DmMRXfil 6 4|eI(0*DS[dS£«^-Cfc^ o ftoT. (8) 

oft QF' I? ©Hl» t ifc^, 5 ffiptfi 1- r t Hi V % Z> „ * ro±, 
<^„ Lfrlste&b. S5|:*i>T, H&##fi, £*g§,«'^ T'tM t'- 
71 — Al — 5fps a>e>jft2 5f P s £1^5 hWTfc<6;T£-£5 



[ 0 0 6 5 ] if m # fflMoM^r h X t- H T >y 7°£ 0I> 



fcAK. tfgsS (11. 13) 0JS*W$rTMjR*ta& 

i j &t«* p i i 2* fc-#ifiv * 2 «m t jfiM § fl 
I). £it£Offflfc2iiWKifi«*-&i:. IfUX < 1 
0, 1 2) cOj«ffi 1 t£Ji?<;t* , >fc, tisybWm 

0.12501651 -0.17338332 0.16332089 ••• -0.03447659 

0.17340284 - 0.24048958 0.22653259 ••• -0.04782041 

0.16334190 - 0.22653624 0.21338904 ••- -0.04504584 



{itl 3] 



0.03449197 - 0.04783635 0.04506013 - -0.00951207 

[&14] 



<46) 



(47) 



fi : ?iJ^#S*5-2c7)'<dft-Sj£u<l^* , >l.fc. ffW 
(4 7) 

0.1250 -0.1875 0.1875 
0.1875 - 0.2500 0.2500 
C„ = 0.1875 - 0.2500 0.1875 

• * • 

• ■ • 

0.0625 - 0.0625 0.0625 

[0066] DCTSSti (-2048 to 2047) WffiHflKA^ 

u, = 0.1250V, -0.1875V, +0.1875v,-0.125Ov 4 +0.1250v,-0.1250v s + 0.0625v, -0.0625V, (48) 

«, =(v, »3)-(v 2 »3)-(v 2 »4)+(v, »3)+(v, »4)-(v 4 »3)+(v 3 »3)-(v 6 »3) + (v 7 »4)-(\fe »4) 



- 0.0625 
-0.0625 

- 0.0625 

* 

0 

[ 0 0 6 7 ] § fefc. aWWU-ZWCtto-C JB«r^/l- 

(48-50) Sr#H3) . 
■IR1 5] 



(49) 



"i = (>, ~ v i + v j - v 4 + V s - v «) » 3+(-v 2 + v 3 +v 7 -v s ) » 4 (50) 
[0068] 



(£0) )03-348598 ( P 2 0 0 3 -ch=tt 



Kftl 6] 



"i = (v, - v, + v, - v,, + v, - v 6 ) » 3 + (-Vj + v, + v, - v g ) » 4 



(51) 



u 2 =( v 3- v j)» 2+ ( v i " v 4 +v 3 -v 6 +v 7 )»3 + (v,-v 4 +v J -v 8 )»4 



(52) 



u, = (v, + v, - v 4 + v, - v tf ) » 3 - (v 3 » 2) + (v, + v, - v 4 + v, + v 7 - v 8 ) » 4 (53) 



w 4 =(v, -v 2 +v 3 -v 4 +v, - v „)»3+(v, - Vj -v 4 +v, -v g )»4 



(54) 



"j =( V 1 ~ V 2 +V 3 ~ V 4 +V J - V «)» 3 +(- V 2 + V 3 +V 7 - V s)» 4 



(55) 



"« =( V l " V 2 + V 3 ~ V 4 +V j) »3+ (V 7 -V 4 ) »4 



(56) 



u 7 = (v, + y, - v 4 + v 5 - v 6 + v7) » 4 + (v 2 ) » 3 



(57) 



[0069] 



"» =( v i - v 2 +v, -v 4 + v s ) » 4 



(58) 



[0070] h'r^Oii^ taKa»TB«Wfc3tfflS*i 

t'r^fSWM 7*5 4 S Z t lfiX'% h . 
[007 1] 2oo*>f7<03l»fT*l. o£ 9, *S5C 
(11) £*Lfc5&B£M&4MtTMi t , ( 1 

3 ) tcSLfc^aXUSHflrrn^iCo^TR^L-C* 

8x8^hJ^frW*Jflvtf:i»£i:JI:*C, ft**** 2 
LfrLfctffc. ¥B3BS*fT*JT* P ifcifi 
ftSSS £ ffigffl w h , T M h p i <05fi(Rff 5>l Sr ffl ^ & 
fc. ¥Big»£*SfltJ4PSNR£<KT£tf& <S8) fc 



ft MBPMk $ tot t'r i KM * 6 £ #6 

m^iih%comzM®x'mz&izm£co%mz£.i-2 
[oo72] *mmnmzmm&mizmmmz%m 

■fhZtX'Zo LlZtR&WW*mfct& Z t 

it**?**, witf. mamafflt* TM hpi £3rr 
xttum&ffm&tjtri. ztihm?m±3 2 t* 
ti&\ zuz, ®mftMimitmmmmzDct7v 



(21) )03-348598 (P200 3-^598 



Al/-h*JftRt*6ii:**IEL"r^6«)t:»L, 3* 



[*8] 



(128 kbps, QCIF, 
ISfps) 


(PSNR_Y) 


^-f 7JJ y K TM 
(PSNR_Y) 


*g®£(PSNRY) 


Sample A 


25.53 


25.53 


22.65 


Sample B 


22.47 


19.57 


18.75 


Sample C 


30.79 


30.66 


29.90 


Sample D 


33.29 


33.25 


28.93 


Sample E 


31.27 


31.10 


28.89 



[0073] 11 6&. #»Btf>l$eSfcWfc:J:4. ffi&S 

•y 1-7. M;-ArtOt*r*r-^7W- A£S<tK£:t 

h U -AtiffiU- h h" -y h X h i; -A?$> 4 . Witf . 
b' y h-Xb'J-Ati. MPEG 4. H.263. H.261&fc'. 4tfD 

<r>umamz^v—^a > 2 e 2 «t^. fitt'^ 

h7h y-A07l^-A<D7a y7;ti<glip-tM 
<DCT)**aSfca*ft;&ft4. ZZX\ b'f'jfti. 
02. 6B. 1 5^Ucr3-r^}:\ r3-r<7)ft 
fcW^Xf-S/TffiaSft*. -T=5ri?*>, b'-ftfr-:? 

fi. ffiHSt'-y hT. h U-A^DCTIHWWHCa-f^W* 

> 2 6 4 tan*, zzx-mmrnkftk mmtm$>tih 

T-?tfj^yy 7 Fr-^flBtfcftfirSft*. MLfc 
A^/iJ -/ Fr-^ffijili. Ill O&tfl 2 tea writ 

■47VvVT-?mmt. mm. -fe/i^-mfg. pd 

[00 74] 9l*tt*. Hi 6fc*$WC, ifltfffili* 
^U-S/HV2 6 6fc:3»», *iTI)CT««WRi:Wi 
ft»t 6 ft * r - ? £EEtliBK-Ci£ll & ffi flt^Jlff S ft 

*. zzx\ mzmmzte, mm9izt5\^xwm 

ffl£ft£. JLc7)*ffi{i<X{C7j-^i^-^Hy2 68{cjt 

», *£TvW7U vh«#&?/giM88;tf. % 
X^& t'«y b7 h U-AOb'-y h-b -y h<7)tp COf^mzJ: 

TWtiunzti&. ymww^wm 

Hco^zit, z o^te*^- x 3 y 2 7 0 tJi 



4 SfctiOTBWLfcJ: die, Bft#K!S£fc:J: 9ff?iJ 

*** { -ji^S5C{cS7c$fti». -r^*., ff^ij^w 

* i fi : ?i|)i^JT'K#^i.^ft-l.„ ^^i/-i/ 3 y2 
6 8 -CSaiffMA%^B*ff «T* 2. t WS6 Sftfc*£ 
CO*a«i^L— 3 y 2 7 2 Cit^, -f-^f 

v^ftS. -I-I-Cli. *S£4 6~5 8K:i3ivc$fcfc:K 
•y KT-^fifitfcJ: OStlRT*^* U^ffiMSrTOc 

•y vmLttM/m&mmzmmiizmt zztiz 
x*). smimx'<?>mmtfmztiz>. 

[00751 H1 7(i. 1 mmbzxh . ;vf 

Tft*. T-fXTl-<Bffi2 80ttiSb'yM^-hbT' 
*tJ: -J-CJg«$ftfcBfll* J: d CflURSftT 

V^S. r^XTl/^BfS280$-. PDA. -feyl^ 

«l^ffi«-i 1 «ifli:Ma#(tl»;i:^-C'#l>. ^-;P2 8 
2(ih*r*-CSiEWd{^»L"CV^. 7o -y ^ 2 8 4 

{i^irfi»^7'y'x^ho>gfflfceaL. KJKX{i4>e 

K<7)tt#««i:#i^ft, 7U-A*^7U-AT?E*) 

-Afrt>yi>-&x'mnmzmttiX'$)&. ^t, 

ffi^b'«y b*hV-J*(OmmmifZ, T-fyU-ACO 
yD-y^2 84(4. 71^-A*^7U— AT'. SS^riJ 
*««i:Waft»t£ft*tf>fcttU 7"D y^286{i7 
lx-A*>A> 7 P-AT'H ; KWfc(S| C4 l> 
&««(tMii#(t^ftl»7'D-y^284J±. 

t s *\ 70 -y ^ 2 8 6 imnmz^h t> tc \ vnxws. 
mcomMmm. 040. mmwmztii. 



(£2) 103-348598 ( P 2 0 0 3 -igffcift 



2 84 tarn u mm&im&rt-vtyyvyvrvv 
tfm&ztii. lmmmx-ii, 02, 6B. i5£*l 

1 0 0 7 6 3 , ft fcUMB LfcXHMttV 7h-)iT 

m^mLtttzthffmmy- yvva hw? r- 
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1. Tit 1 « of Invention 

Method and Apparatus fcr Memory Efficient Compressed Domain Video Prcces 
sing and fcr Fast Inverse Met i c n Compensation Using Factorization a c d la 
J e f e r Approximation 

2. Claims 

I. A method for reducing tbe memory reqrji i eme n t s Icr decoding a bit stre 

am. ccmpri s ing: 

receiving a video bit stream; 

decoding 3 frame cf the bit stream into a transform domain r c p r cs e o ta t i c 

n ; 

idctiifriog ccd-2€ tg coefficients of the transform dcmaio representation 



assembling a hybrid data structore including a fixed sire array and a ra 
r i a b 1 c size ore r \ i g* Tec t or ; and 

inserting tbe non-re re coefficients cf tbe transform domain represent ati 
en into tbe hybrid data strcctcrc. 

2. Tbe method cf claim 1, wherein tbe video bit stream is a lew rate vid 
cc bit stream. 

3. Tbe method of claim I, Therein the method cperaiico cf decoding a fra 
mc cf the bit stream into a transform demain representation includes, 
processing tbe bit stream through a variable length decoder and a dequan 
t i i a t i c n block. 

4. The method cf claim 1. therein the fixed siie array ioc lodes fixed si 
ze blocks. 



* ! 
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5. The metbcd c( claim < wherein the fixed sixe blccis arc ccafigured to 

store 8 ncn-iero coefficients cf the transform domain r cpr eseatat ion . 
5- The metbcd of claim 1 T therein the method operation cf inserting the 
ncn-ierc coefficients cf the transform domain represectit ion into the fay 
brid data structure includes, 

mapping coefficients in the fixed sire array tc cc r respond i ng coefficien 
ts in tbe variable size cverflc? vector fcr each blcck cf the frame. 

7. A method fcr decoding video data, comprising: 

receiving a frame cf video data within a compressed bit stream; 
decoding a blcck cf the frame into a transferal dcmaiD r epr c secta t i co -i a 
the compressed demai n: 
defining a hybrid data structure; 

storing data associated with the transform domain re presents i i en in the 
h r b r i d data strpctar«: 

performing inverse motion compensation cn the data asscciited with the t 
raosfcrm domain representation in the compressed domain; and 
after performing the inverse motion ccmpensa t i cc cc the data, decompress 
i og t be da t a f c r d i s p 1 a y . 

8. The method cf claim 7, wherein the hybrid data structure includes a f 
i x e d size array cf fixed sue blocks and a variable size c v e i f 1 o vector 

9. Tbe method cf ciaim 7, wherein the me i hod operation cf storing data a 
sscciated with tbe transform domain representation! is the hybrid data st 
r t c t d r e includes, 

identifying acn-zerc coefficients cf the transfcrm domain representation 

storing the ncc-zerc coefficients intc a fixed sire block cf the fixed s 
ize array cf the hybrid data s I f u c 1 k r e until a capacity cf rhe fixed s i a 
e blocks is reached; and 
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after reaching the capacity cf the fixed size blocks, storing ncc-zerc c 
cefficients exceeding the capacity cf the fixed size blocks in at overfl 
cw vector. 

10. The method cf claim 7, wherein the ccmpiessed bit stream is a lew ra 
t e bit stream. 

11. The method cf claim 7, wherein the methcil operation cf performing in 
verse motion compensation on the data associated with the transform dema 
in representation in the compressed domain iochdes, 

applying a hybrid factor i rat ion and integer approximation technique i o I 
be data associated with the transform domain representation. 

12. A computer readable media having program indirections for rearranjio 
g lew rate bit stream data fc r storage into a hybrid data structure, com 
p r i s i u g : 

program instructions for identifying ncn-rerc transform coefficients ass 
cciated with a coded blcck cf a frame cf data; 

prGgram instructions for arranging the ti c n - z e r c transform coefficients i 
n t c a fixed siie array; 

program institutions for determining if a quantity cf the ncn-zere iracs 
form coefficients exceed a capacity cf the fixed size amy; 
program instrncticos for storing the octi-zero i r a ns f c rm c c e f f i c i e n t s exc 
eeding the capacity cf the fixed site array in a variable sire overflow 
UCtor; tod 

program iastrncttcos for translating the ocn-zerc transform coefficients 
from a compressed domain tc a spatial domain. 

13. The computer readable media ol claim 12 ; wherein the fixed size aira 
y includes a piu/aiity cf fixed size blocks. 

14. The computer readable media cl claim 13 ; wherein each cf the fixed s 
ize blocks arc configured to store a maximum of eight bc n - 2 e r o transform 

coefficients. 



« • 

♦ I 



(61) )03-348598 ( P 2 0 0 3 -ffil <§tt 



15. The ccmpEtei readable media cf claim 12, fujthcr including: 
program instructions fci mapping coefficients in the fixed size array tc 

corresponding coefficients in the variable siic overflew vector fcr eac 
h block of the frame of data. 

16. The computer readable media cf claim 12. forth c r including: 
program instruction? fcr performing inverse motion compensation cn the o 
c b -i € r c transform coefficients through the application cf a hybrid facte 
riiaticD acd integer approximation technique. 

17. A circuit, compr i s i ng : 

a video decoder integrated circuit chip, the video decoder integrated, ci 
rcuit chip including. 

circeitry fcr receiving a bit stream cf data associated vith a frame cf 
t i d c c data; 

circuitry fcr decoding the bit stream cf data into a transform domain r 
e p r e s e a t a t i c o ; 

circuitry fcr arranging ocn-zeic transform coefficients cf the transform 
domain representation in a hybrid data slrsctire in a memory associated 
with the video decoder ; and 

circuitry for decompressing the non-zero transform coefficients cf the 
transform domain representation fcr display. 

IS. The circuit of claim 17. therein the bit stream is a H.263 bit strea 
m. 

19. The circuit cf claim 17, wherein the memory is separate from the vid 
ec decoder integrated circuit chip. 

20. The circuit cf claim 17. further inclcding: 

circuitry fcr performing inverse motion ccmpensatico thiongh a hybrid fa 
cicritatlcnand integer approximation technique. 

21. The circuit of claim 17. wkereiu the memory is a static random acces 
s memory. 
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22. A device coof i^oied tc display a video image, ccmpr i s t ng : 
a central processing no i t (CPU] ; 

a r and cm access memory (KA.Vj) ; 

a display screen configured tc present an image; 

decoder circuitrf configured tc transform a video bit stream into a tran 
sfcrm domain representation, the decoder circuitry capable cf arranging 
non-zero transform coefficients cf the transform domain representation i 
n a hybrid data structure io a memory associated 7 i t b the decoder circui 
try. the decoder circuitry including cirtHitry for selectively applying 
a hybrid f ac t cr i 2 a t i cn/i o t eger a p pros ima t ion technique during inverse - mc 
t i cn ccmpcnsaiicn: and 

a bus in c ommu o i c a t i c & with, the CPU, the RAM, the display screen and the 
decoder c iicutjy. 

23. The device ol claim 22. wherein the device is a pctable electronic d 
€ v i c c . 

24. The dence of claim 23, wherein the portable electronic device is se 
lected from Ibe group consisting cf a personal digital assistant, a cell 
ular phone, a web tablet aod a pocket personal computer. 

25. The device cf claim 22 ? wherein the hybrid data structure includes a 
fixed size array having a -plurality cf fixed sire blocks and a variable 
s i 2 c overflew ? e c tor . 

26. The device cf claim 25, wherein each of the pinraiity cf fixed size 
blocks arc configured tc hold 6 ncn-zerc transform coefficients. 

21. The device cf claim 26, wherein non-zero t r an s f o rmc ce f f i c i e n t s in ex 
cess cf 8 are stored in the variable size overflow vector. 

28. The device cf claim 22, wherein the decoder circuitry includes an cd 
-chip memory configured tc store data associated with the hybrid data st 
r Dcture . 

29. The device of claim 22, wherein the circuitry for selective!? applyi 
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Dg a hybrid factorization/integer approximation technique daring inverse 
roc t i o o ccmpcos a t i oo i oc lade 5 , 

circuitry fcr identifying blocks cf a frame cf the video image as being 
associated with cue cf an active motion area and an inactive motion area 
; acd circuitry fcr per form log inverse mctico compensation by applying a 
factorization technique to the blocks associated with the active motion 
area and an integer approximation technique tc the block? associated *i 
tb the inactive motion area. 

30. The device cf claim 22. wherein the video bit stream is a lew rate i 
i dec bit s 1 1 earn. 

31. h method fcr performing inverse memory compensation, comprising: 
receiving a video bit stre am; 

identifying a transform matrix type selected free the groip consisting c 

* 

f a half piiel matrix and a foil pixel matrix: 

if the transform malm type is a half pixel matrix, the methes includes 



applying a factor iiatici technique to decode the bit stream ecrrespendi 
ng tc the half pixel ma t r ix; and 

if the transform matrix type is a fall pixel matrix, the method includes 



applying an ioteger approximation technique tc decode the hit stream cc 
irespending to the lull pixel matrix. 

32. The method of claim 31, wherein the video bit stream is a lew rate v 
i de o bit stream. 

33. The method g1 claim 31, wherein the method operation cf applying the 
factor hat icn technique tc decode ihe bit stream corrcspoodicf tc the b 

a 1 f pixel mat r ix i nc 1 Dries . 

factoring the half pixel matrix intc a sequence cf sparse matrices, the 
sparse matrices including permEtaticn matrices acd diagonal matrices. 



» • 
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34. The method cf claim 31, wherein the method operation cl applying an 

s d t e g e r apprcxiraat ion technique tc decode the bit stream cgt rcspcod iug t 
c the full pixel matrix includes, 

approximating each element cl the full pixel matrix silk binary numbers. 

35. The method of claim ;H, wherein earh element is rounded ic a nearest 
power cf twc. 

36. A method for decoding video data, comprising: 

receiving a frame cf v i dec data within a compressed bit stream; 
decoding a block cf the flame into a transform domain representation in 
the compressed domain; 

storing data associated with the transform domain representation ic a by 
b r i d d a t a s t r a c t a r e ; 

p e r f c r m i n g i o v e r s e motion compensation on the data associated with the t 
ransform domain representation in the compressed domain; the performing 
inverse motion compensation including. 

determining a type cf transform matrix associated with a portico cf the 
f rame cf ? i dec data ; and 

applying a hybrid factorization and integer appr cxima t icn technique tc 
enhance inverse mctico compensation. 

37. The method cf claim 36. -wherein the ccmpressed bit stream is assccia 
ted with a standard selected from the group consisting of H.263, 3.261 a 
nd Motion Pictures Expert Gt c u p . 

38. The method cf ciaim 36, wherein the hybrid dala structure includes a 
fixed size array and a fariable siie ever flow vectcr. 

39. The method of claim 36, wherein the type of uansfcrm matrix is selc 
ctcd from the group consisting cf a half pixel matrix and a full pixel ra 
a t r i x . 

<0. The method of claim 39, wherein the half pixel matrix is associated 
with a high motion region cl an image and the full pixel matrix is assoc. 
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iatcd with a minima! motion region of the image. 

41. The method cf claim 36 r wherein the net bed operation of applying a h 
ybrid f a c t o r i i i t i o d and integer approximation technique to enhance infer 
se motion c crape d s a t i c o includes , 

applying a factorization technique tc mat j ices associated with blocks cc 
riespccdiDg tc bi^h mciicn regies cf the frame: and 

applying so integer apprcx ireat ice technique tc remaining blocks cf the f 
r a me . 

42. The methed cf claim 36, wherein the compressed bit stream is a )c* r 
ate bit stream. 

4 3. A computer readable media having pre gram instructions for performing 

inyerse motion com pens a t i c.n in a compressed domain, comprising: 
program instructions Icr identifying a transform matrix; 
program i ns i r a c t i o as lor determining if the transform matrix is cne cf a 

half piiel matrix and a fell pixel matrix; 
program instructions for applying a factorization technique tc decode bl 
cch cf ihe bit stream corresponding tc the half pixel matrix; and 
program instructions for applying an integer approximation technique to 
decode blocks cf the bit stream corresponding tc the foil piiel matrix. 

44, The computer readable media cf claim 43, wherein the program instrnc 
tiens for performing inverse motion compensation is executed in the cenp 
ressed domain. 

45, The ccmpnter readable media cf claim 43. further including: 
program instruct iens for extracting motion rector data, the motion vectc 
r data identifying the trans form mat r ix as one cf the half pixel matrix 
and the full pixel matrix. 

46, The compete! readable media of claim 43, further ioclediog: 
program instructions for arranging n c c - 2 e r c transform coefficients as sec 
iated with a coded block of a frame of data into a h 7b rid data structure 
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47. The computer readable media of claim 43, wherein the program instioc 
tiens fcr applying an integer apprcximaticn technique to decode blocks o 
f the bit stream ecrrespending to the full pixel matrix iodides, 
program instructions fcr apprcximating each element d1 the fell pixel ma 
tris with binary numbers. 

48. The computer readable media cf claim '13. Therein the prcgram instruc 
t i c n $ fci applying a factorization technique tc decode blocks cf the bit 

stream ccrrespcndicg tc the half pijeel matrix includes, 
program iostrcctiens fcr factoring the halt pixel matrix iotc a sequence 

cf sparse matrices, the sparse matrices including permutation matrices 
anil d iagena 1 ma tr ices . 
4 9 . A circuit, ccmpr i s i ag : 

an integrated circuit chip configured tc decode video data, the integral 
ed circuit chip including, 

circuitry fcr receiving a bit stream cf data associated with a frame cf 
video data; 

circuitry fcr decoding the bit stream of data into a transform domain i 
epresentat icn; 

circuitry fcr identifying -a type cf transform matrix; and 
circuitry for performing inverse met let compensation thrcuRli a hybrid f 
actorization and integer apprcximaticn technique. 

50. The circuit of claim 49. wherein the integrated circuit chip further 
includes: 

circuitry fcr arranging non-iero transform coefficients cf the transform 
domain representation in a hybrid data structure. 

51. The circuit of claim 49, wherein the bit stream is a low rate bit st 
ream. 

52. The circuit of claim 49, wherein the circuitry (or performing infers 
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e met i c d compensat icn through a hybrid facte riiat icd and integer apprcxi 
(nation technique is configured tc apply a factor L2aL ion Iccboiqcc tc a h 
alf pixel tran?fcrm matrix and an integer a p p r ox ima t i o n technique tc a f 
u 1 1 pixel transform matrix. 

53. The circuit cl claim 49, further including a nemexj in cc annuo i ca i i oo 
with the integrated circuit e k i p . 

54. The circcit cf claim 49, wherein the hybrid factorization and intege 
r approximation technique if applied tc data in the compressed domain. 

55. A video decoder, comprising: 

a variable length decoder (VLD) configured tc extract coefficient values 
and mctico vector data ircm an incoming bit stream; 

a de quan t i za I i c n block in coram* n i ca t i c o with the VLD, the dt q o an t i z a t i c o 

block ccDfiguicti to rescale the coefficient values: 
a lc^er branch io communication with the den. D3 d t i za i i en block, the lower 

branch configured tc decode error coefficients into a spatial domain; a 
od 

an upper braocn in common icat i c o with the dequant izati cc block, the uppe 
r branch configured tc maintain an internal trans fcim domain rcpreseotat 
ion, the upper branch configured tc generate a spatial domain output cap 
able of being added to the-deccded error coefficients tc reconstruct a c 
urrcm block. 

56. The video deccder of claim 55. wherein the video decoder is implemen 
ted in software. 

57. The video deccder cf claim 55, wherein the video decoder is implemen 
ted in hardware* 

58. The video decoder cf claim 55, wherein the incoming bit stream is a 
low rate bit stream. 

59. The video deccder cf claim 55, wherein the upper branch includes a f 
eedback loop, the feedback leep including a frame buffer, a motion enmpe 
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usance block and a discrete cosine transform block. 

66. The video decoder of claim 55, Therein the lever branch include* a r 
no length decode block and ac inverse transform blcck. 

61. The tideo decoder cf claim 55. wherein inverse net ico compensation c 
peraticiu are pci fcimcd Id a compressed domain. 

62. The videc decoder cl chic 55, therein nco-itro coefficients cf the 
transform domain representation are arranged in a hybrid data strcctnrc 
in memcry associated with the video decoder in order tc rednce memory re 
qu i r erne n t s . 
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echcique tc a balf piicl t rans f crm matr ii and an integer a pr> r c* ima t i on t 
ec hoi que tc a full piiel transform matrix. 
3 .Ba ckgrcond of the Invention 
Field of the Invention 
This invention relates generally tc digital video technology a o d mere p 
articolarly tc a method and apparatus for implementing efficient memcry 
compression methods and tc a method and apparatus for implementing effic 
ient inverse motion ccmpensat ion methods for a ccmpressed domain video d 
eccder . 

Description of the Related Art 
The access cf video cn mobile terminals, such as cellular phones and pe 
rscnat digital assistants, presents many challenges be cans c cf the limit 
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atioDS due to the nature cf the mcbiie systems. For example, lcv-pc^crc 
d , handheld devices are constrained tinder bandwidth, power, memory, and 
cost requirements. The video data received by these handheld devices ar 
e decoded through a video decoder. The video decoders associated with 5 
ucb terminals per form met i cn compensation in the spatial domain, i.e., i 
eccropressed domain. Yidec c crop re s s i on standards, such as H.263. H 2 6 1 an 
d MPEG- 1/2/4 , tse a met icc-ccmpcnsatcd discrete cosine transform (DCT) s 
cherae to encode videos at lev bit rates, As used herein, !c* bit rates 
refer to hit rates less than about 64 kilobits per second. The DCT sche 
me uses motion estimation (ME) aod mction compens a t i on (MC) to remove - te 
mccral redundancy and DCT tc remove the remaining spatial redundancy. 

Figure 1 is a schematic di.agram cf a videc decoder let decoding video d 
ata and performing motion compensation in the spatial domain. Bit strea 
m 102 is received by decoder 100. Decoder 160 includes variable length 
decoder (VLD) stage 104, run length decoder (RL3) stage 106, Deqoantizat 
ion (DQ) stage 10S, inverse discrete cosiae traoslcrm ( F DCT) stage 110, 
motion compensation (MC) stage 112 and memory (ME.M) 111. also referred t 
c as a frame buffer. The first four stages (VLD 101, RLD 106, DQ 108, a 
cd IDCT 110) decode the compressed bit stream back into the pixel domain 
. For an intraccded block.- the output cf the first four stages, 1C4, 10 
6, 108 and 110, is used directly tc recoostroct the block in the cerreot 

frame. For an interceded block, the output represents the prediction e 
ircr and is added tc the prediction formed from the previous frame tc re 
construct the block in the current frame. Accordingly, the current f r am 
e is reconstructed cn a block by block basis. Fiaally, the current f r am 
e is sent tc the output of the decoder, i.e., display 116. aod is also s 
1 0 red in frame buffer (MEM) 114. 

MEM 1H stores the previously decoded picture required by mction corn-pen 
sat ion 112. The siie cf MEM IU must scale with the incoming picture fo 
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roiat. Fcr example, H.263 soppcrts five s t aoda r d i led picture formats: (l 
) Sttb-quai ter common intermed iate format, (sob QCIF), (2) quarter common 
intermediate format (QCIF), (3) common intermediate format (C1F), (4) 4C 
IF, and (5) 16C1F. Each format defines the width and height of the pict 
urc as well as its aspect ratio. As is generally known, pictures are ce 
ded as a single luminance ecropeneot and etc color difference components 
(i\Cr,Cb). The ccmpcoents are sampled in a 4:2:0 cenf i go ia t i on , and eac 
b component has a resolution of 8 bits/pixel. Fcr example, the video de 
coder cf Figure 1 must allocate approximately 200 kilobytes cf memory fc 
r MEM 114 while decoding a H.263 bit stream with Clf format. Furiheimcr 
e, when multiple bit streams axe being decoded at ence, as rep i red by v 
idee cenf e reoc i ng systems, the demands fcr memory become excessive. 

MEM 114 is the single greatest source cf memory usage in video decoder 
100. In order re reduce memory usage, one approach might be ic reduce t 
he resolution cf the ecler ccmpcoents fcr the incoming bit stream. For 
example, if the cofcr display depth cn the mobile terminal can enly shew 

65,536 colors, then il is possible tc reduce the resolution of the cole 
r components (Y,Cr,Cb) from 24 bits/pixel down tc 16 bits/pixel. While 
this technique can potentially redece memory usage by 3 05c , it is a displ 
ay dependent solution that must be hardwired ic the videc deccder. Alsc 
, this technique docs not scale easily with changing peak signal-tc-ncis 
e ratio (PSNR) requirements, therefore, this approach is net flexible. 

Operating on the data in the spatial domain requires increased memory c 
apacity as compared to compressed domain processing, in the spatial d cm 
ain. the motion compensation is readily calculated and applied tc Sicces 
give frames of an image. However, when operating in the compressed dcroa 
in mcticn compensation is net as Straight forward as a motion Fector pcin 
ting back tc a previous frame since the error values are oc longer spati 
a I values, i.e., the erior values are not pixel values when operating in 
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the compressed domain. Add i 1 i c oa I I y , methods capable of efficiently ha 
ndliog compressed domain data are cot available. Prior art approaches h 
ave fccised mainly on transcoding, seal i a g and sharpeoiog compressed dem 
ain applicaticos. Additionally, inverse compensation applications for t 
he compressed demaio. tend to give poor peak signal to ache ralic (PSNR) 

performance and at the same time hate an uoaccepiably slew response tim 
e in terms of the amennt cf frames per second that can be displayed. 

As a result, there is a need to solve the problems cf the prior art to 
provide a me t h c d and apparatus that minimizes the demands on memory for 
decoding lev bit rate video data and tc provide a methed and apparatus t 
c enable fast and efficient inverse mctic.o compensation for a compressed 

d cola ic video recorder. 
Simma ry c f the Invention 

Broadly speaking, the present invention fills at least one aspect cf ib 
ese needs by presiding a video decoder configured tc minimize the memor) 

requirements tbrcngh the use cf a hybrid data structure. It should be 
appreciated that this aspect cf the present invent ion can be implemented 

in rnrcercas ways, including as a method, a system, computer readable me 
dia cr a device. Several inventive embodiments cf this aspect of the pr 
esent invention are described be lev. 

fn one embodiment, a methed for redncing the memory r e q o t r erne n t s for de 
coding a hit stream is provided. The method initiates with receiving a 
video bit stream. Then, i frame of the bit stream is decoded into a tra 
nsfcrm (e.g., a discrete cosine transform (DCD) domain repr eseota t ion . 
Next, nco-zerc coefficients cf the transform demain representation are 
identified. Then, a hybrid data strncinrc is assembled. The hybrid dat 
a structure includes a fixed size array and a variable size overflow v e c 
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tor, Neil, the oco-zcro coefficients cf the r r a d s f c rm domain represeota 
tion i r c inserted i a t c tbe hybrid data stroctnre. 

In anctber emb c d i iu n t , a method fci decoding videc data is provided. T 
he method initiates with receiving a frame cf videc data within a c crap i c 
sscd bit stream. Then, a block cf the frame is decoded into a transform 

(e.g., DCT) domain representation in the compressed domain. Next, a by 
brid data strjetire is defined. Then, data associated with the transfer 
m domain representation is stored in the hybrid data structure. Next, i 
nvcrsc motion compensation is performed cn the data associated *ith the 
tracsfcira domain representation in the compressed domain. After perform 
ing the inverse motion compensation cn the data, the data is decompress 
d icr display. 

Id yet aocther embodiment, computer readable media having program instr 
nctiGOS fci rearranging low rale bit stream data for storage into a hybr 
id data structure is provided. The computer readable media inclndes pre 
gram injunctions for identifying non-zero transform U-g-. DCT) coeffic 
ients associated *itb a ceded bled cf a frame cf data. Program ins true 
tions for arranging the non-zero transform coefficients inic a fixed siz 
e aria? are included. Program instrnctiGns for determining if a qnantit 
y of the non-zero transform coefficients eiceed a capacity cf the fixed 
size array are provided. Program instructions for storing the ncn-zerc 
transform coefficients exceeding the capacity of the fixed sixe array in 

a variable sire ever flow vector and program instructions for trans latin 
g the non-zero transform coefficients from a compressed domain tc a spat 
ia I domain are included . 

In still yet another embodiment, a circuit is provided. The circuit in 
c lodes a tidec decoder integrated circuit chip. The video decoder integ 
rated circuit chip inclndes circuitry for leeching a bit stream of data 

associated in t h a frame cf yiden data. Circuitry fur decoding the bit 
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stream cf data into a transform (e.g., DCT) domain representation is inc 
laded in the videc decoder. Circuitry f c r arranging non-ierc tiacsfcrm 
coefficients of the transform d cm ain representation in a hybrid data str 
ccture in a menic r y associated with ibe ridec decoder is provided. Circo 
itiy fcr decompressing ibe nco ierc transform c c e f f i c i e o t s cf the transf 
o r id domain representat i on fcr display is alsc prcrided. 

in another embed imeot , a device configured to display an image is picvi 
ded. The device incUdes a central processing o n i t (CPU), a Tandem acce 
ss memory (RAM) , and a display screen cenfignreo tc present an image. D 
ecoder circuitry configured tc transform a videc bit stream into a trans 
fcim (e.g., DCT) domain reprcseotaticn is included. The decoder circuit 
ry is capable dI tiring jog .ocn-ier o trace fcrm coefficients the transf 
crm domain representation in a hybrid data si met ore in a memory as sec i a 
led with the decoder circcitry. The decoder circuitry includes circniir 
y for selectively applying a hybrid lac tc r i i a t i cn/i o reger apprexima t i c n 
tecboiqQe during inverse motion compensation. .A bns in ccmmunicaticn wi 
th the CPU, the RAM, the display screen and the decoder circuitry is als 
c included. 

Bread! y speaking, the present inTenticn fills at least another aspect c 
f these needs by p f o t i d i o i a video decoder capable cf performing inverse 

motion compensation in the compressed domain while reducing memory reqn 
irements and provide acceptable videc qoality. It s hen id be appreciated 

that this aspect cf the present indention can be implemented in nomercn 
s says, including as a method, a system, computer readable media cr a de 
nee. Scleral in v entire embodiments cf this aspect cf the present i o v < n 
tion are described be 1 c\* . 

In one embodiment, a method fcr performing inverse memory compensation 
is provided. The method initiates with receiving a videc bit stream. T 
hen, a t r ans f c im ma t r i i type is identified. The trans form mati ii type i 
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s either a half pixel matiix cr a full pixel matrix. If the t r an s f cira m 
airix type is a bal( pixel matrix, then t b c met bed includes applying a f 
actciizatioo technique tc decode the bit stream ccrrespcod iog tc tbe hal 
f pixel matrix. )( tbe transfcxm matrix type is a full pixel matiix. th 
en the method includes applying as integer approximation technique tc de 
cede the bit stream corresponding tc tbe full pixel matrix. 

In another embed ime at , a method lor decoding riileo data is provided. T 
he method initiates ffith receiving a frame cf video data within a c cap r e 
ssed bit stream. Then, a block cf tbe frame is decoded into a transform 

(e.g. r a discrete cosine transform (DCT)J domain representation in tbe 
compressed demain. Next, data associated with the transform domain rep; 
esentaticn is stored in a h.ybrid data juncture. Thee, inverse motion c 
cmpensa t ion is performed cn tbe data associated with tbe transform domai 
o representation in the compressed domain. Determining a type cf transf 
crm matrix associated with a portion of the frame of video data, and app 
lying a hybrid factorisation and integer app r cx ima t i cn technique tc enba 
nee inverse motion compensation are included in performing tbe inverse m 
o t i cn c cmpensa i ion. 

[0017] Is yet another embodiment, a computer readable media having prcgr 
am instructions f c r perforating inverse mctico compensation in a ccmpress 
ed demain is prcrided. The computer readable media includes program ins 
tractions for identifying a transform matrix. Program instructions for 
determining if the transform matrix is either a ball pixel matrix oj a f 
nil pixel matrix are included. Program i n s t rte t i crjs for applying a fact 
orixaticn technique to decode blocks cf the bit stream corresponding to 
the balf pixel matrix and program instructions for applying an integer a 
pprcximaticd technique tc decode blocks cf the bit stream corresponding 
tc the full pixel matrix are included. 
In still yet another ecbediment. a circiit is provided. The circuit in 
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eludes an integrated circuit chip ccofigured tc decode video data. The 
integrated c i r c d i t chip i d c 1 1 d e s circuitry for r e c e i t i o g a bit stream c f 
data associated with a frame cf video data. Circoitry f c r decoding the 
bit stream cl data intc a transform (e.g., DCT) domain r epr esen I a I ion i 
s included cn the integrated c i r c a i t chip. Circuitry for identifying a 
type cf transform matrii and circuitry for performing inverse mcLico ccra 
pensaticn thrcagh a hybrid f ac 1 cr i z i t i c d and integer apprcximat ice lechn 
ique are provided en the integrated circuit chip. 

In ancther embodiment, a videc deccder is provided. The videc decoder 
includes a variable length deccder (VLD) configured tc extract cccfficic 
oi values and roctico vectcr data from an incoming bit stream. A deyaant 
izaticn block in c ommin i c a t i on with the VLD is included. The deqaaBiiza 
ticn blcck is ccofigoieal tc re scale the coefficient values. A fewer bra 
BCh in communication with the dequaotizatioo block is provided. The lew 
er branch is configured tc decode error coefficients into the spatial dc 
main. An epper branch in communication with the dequanLizaticn block is 

included. The upper branch is configured tc maintain an internal trans 
fcim (e.g., DCT) demaio representation. The upper branch is further con 
figured tc generate a spatial domain output capable of being added to to 
e decoded error coefficients- tc reconstruct a car rem block- 
Other aspects and advantages cf the invention will become apparent from 
the following detailed description, taken in conjunction with the acccm 
panying drawings, illustrating by way cf example the principles cf the i 
nvection. 

Detailed Description of the Preferred Embodiments 

Ao invention is described for a system, apparatus and method fcr minimi 
xing memory capacity for compressed domain videc decoding, It will be a 
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pparent, however, tc cne skilled in the a r t T in flew cf the f clicking de- 
scription, that the present invention may be practiced without seme cr a 
il cf these specific details, fti ether instances, well known process cp 
eratioQS hare net teen described io detail in order net tc on n ece s s a r i I y 

obscure the present i o v entice. Figote ] is described in the " B a c k g r c u o 
d cf the Invention" section. The term abest as used tc herein refers tc 

t/- 1 C% cf the referenced value. 

The enbed iotot s described herein provide data structures that enable th 
e reduction cf the meracrv used while deccding video data in the compress 
cd domain. Id one embodiment, the videc deccding pipeline is rearranged 

such thai the current frame is stored, and the inverse mcticn ccmpensar. 
icn is pcifc/mcd, io tht freqeency domain, i.e., compressed • dema in . Hyb 
rid data strictnrcs a 1 1 c ^ fcr the manipulation cf the data in the c cm pre 
ssed domain wilhcat ccmpe t a t i cna I cost cr an? significant less cf data. 

fa eoc embodiment, the hybrid data structires take advantage cf the lac 
t that there arc only a small number of ncn-rerc discrete ccsine tiansfc 
rra (DCT) coefficients within a ceded block. Tdbs : enly the tnn-ierc DCT 

coefficients of the entire frame are st.cred : thereby redecing the menicr 
y requirements. As will be explained in mere detail belcw, the hybrid d 
a la structure includes a fixed sire array and a variable size overflew t 
ectcr. The variable size overflew vectcr stores the ncn-;erc DCT ccelfi 
cients of the coded blocks that exceed the capacity cf the fixed size ar 
ray. 

Figure 2 is a schematic diagram cf a video decoder arranged such that i 
averse mcticn compensation is performed in the compressed demain in accc 
rdance with one embodiment of the invention. Here, bit stream 122 is re 
ceived by video decoder 120. The first two stages variable length decco 
er (VLB) stage 124 and de qua n t i iz t i on (DQ) stage 126. accede the ccmpres 
sed bit stream iotc a DCT demain representation. The DCT domain represe 
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otatico is stored in memory (MEM) 130, also referred to as a frame biffc 
r, for dsc in notice c omp ens a t i c n (MC) stage 128. Rm length decoder (I 
LD) stage 132 and io?erse DCT (IDCT) stage 134 is perfcinied after the no 
ticn compensation feedback leep which contains AiC 128 and MEM 130. Thus 
, the interna! representation c I the block being decoded is kept in the 
compressed domain. There are only a small Gamber cf nonzero DCT cceffic 
ients within a coded block, therefore, this characteristic can be explci 
ted by deTelcping data structures for HEM 130 that store only the ocizei 
c DCT coefficients cl each block in the frame. As will be shown in mere 
detail be low, the memory compression enabled through the hybrid data st 
mctores can reduce memory usage by 5 C'5t without any less in fide ft qualii 
y. Since the human nml system is more sensitive to the fewer cider D 
CT coefficients than the higher order DCT coefficients, thresholding sch 
ernes that litter cut higher order DCT coefficients and tradeoff memory o 
sage versos changing power cr peak signal to ncise ratio (PSNR) required 
ents are developed as described below. 

Accordingly, a complete compressed domain video decoding pipeline that 
is optimixed for both last and memory efficient decoding is described he 
rein. In one embodiment, TELENOR 5 s ridec decoder, which is a pablic dem 
ain H.263 compliant decoder, is used for the testing referred ic herein. 

It sbcnld be appreciated that while seme cf the embodiments described 
belcw refer tc a H.263 bit sueam. the embodiments are net limited to cp 
erating en a H.263 bit stream. That is. any DCT based compressed bit st 
ream having ndec data, e.g., Mctico Picture Expert Group (MPEG) l/2/4, 
H.261, etc. may be employed. A number cf last inverse motion ccrapensati 
on algorithms tor the discrete ccsioe transform (DCT) domain representat 
ion enable the efficient processing in the compressed domain, it should 

be appreciated that memory compress ion methods thai store the nonzero D 
CT coefficients within a coded block allow for the reduction in memory r 
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eqoirccncots due tc th( compressed demain prccessing. Additionally, pcif 
crmaoce of the video deccder using c omp t e s e c d domain processing with tic 
inverse mc t i c o compensation techniques and mcroc i y compress ico described 
herein is evaluated along three d imens i ens : computational complexity, tn 
cue r r efficiency, mi PSH, tc shew the tar ices performance tradeoffs in 
optimizing fcr b c 1 b speed and memory. 

Figure 3 is a schematic diagram illustrating inverse motion 
compensation as peformed in the spatial domain. Here, a 
prediction of the current block is performed from motion 
compensated blocks in the reference frame. The current 8x8 
spatial block, f k 142, of current frame 140 is derived from four 
reference blocks , f[ , // , and f A , 144-1 through 144-4, 

respectively, in reference frame 146. The reference blocks are 
selected by calculating the displacement of f k by the motion 
vector (Ax, Ay) and choosing those blocks that the motion vector 
intersects in the reference frame. For (Ar>0,Ay>0) , f k is 
displaced to the right and down. From the overlap of /^with 
we can determine the overlap parameters (w,A) and also the 
parameters (%-w,h) , (w\8-A) , and (8-w,8-A) with the neighboring 
blocks. 



4 




(2) 



(14 9) )03-348598 (P2OO3-B098 



Since each block can be represented as an 8x8 matrix, the 
reconstruction of matrix f t can be described as the summation 

of windowed and shifted matrices , / 4 \ In equation (Eq.) 
(2), the matrices^., i = 1,...,4, j = 1,2, perform the windowing 
and shifting operations on/'. The matrices are sparse 8x8 
matrices of zeroes and ones. Also, c tf is a function of the 
overlap parameters (w,/?)and is defined as 




(3) 




(4) 



where / A and /„are identity matrices of dimension h x h and w x 



w, respectively. Similarly, 
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The inverse motion compensation in the DCT-domain 
reconstructs tntracoded blocks from motion compensated 
intercoded blocks. The concept is similar to the spatial domain 
except that all coefficients are kept in the DCT-domain, i.e. 
reconstruct/;, the DCT of/*, directly from F{ , ... , F 4 ' , the DCT 
of/i\...,/ 4 \ 

S is defined as a matrix that contains the 8x8 basis vectors for 
a two-dimensional DCT. Using the unitary property of the DCT 
transform, S'S^I , it can be demonstrated that Eq. (2) is 
equivalent to 

A-it^S/P'Sc*' (7) 

Premultiplying both sides of Eq. (7) by S , and postmultiplying 
by 5", results in: 

r*=i:c,r,c a , • (8) 



(61) >03-348598 ( P 2 0 0 3 Sfflfi 

where C tj is the DCT ofc^. Eq. (8) calculates F k as a summation 
of pre- and post-multiplied terms F{, F 4 \ The matrix C 0 is a 
single composite matrix that contains the sequence of 
transformations: inverse DCT, windowing, shifting, and 
forward DCT. Thus, Eq. (S) describes a method to calculate F k 
directly from F x \ .. , F 4 ' using only matrix multiplications. These 
matrix multiplications operate in the DCT-domain without 
having to explicitly transform between the spatial and 
frequency domains. However, the matrix multiplications 
described are unacceptably slow. In turn, only about 5 frames 
per second can be displayed which results in a poor quality 

■ 

display. The DCT-domain inverse motion compensation 
algorithms described below focus on reducing the 
computational complexity of these matrix multiplications as 
the matrix multiplications become a bottleneck causing 
unacceptable delays. 

Lc7 bit rate video, i.e.. video data banog bit rates less lhac about 6 
4 kilobits per second, is targeted lor applications such as wireless nd 
eo on cellular phones, pergonal digital assistants PSAs, and other haodh 
eld cr battery operated devices, as veil as being osed for video coofeie 
ociog applicaticns. The fl.263 standard is an exemplary standard that sp 
ecifies the bit stream svnux and algorithms fcr video ceding at iov bit 

rates. Tbe algorithms include transform ceding, motion cs t ima l i co/ccmp 
ensatioo, coefficient quantization, and ion-length ceding. Besides the 
baseline specification, version 2 of the standard also supports sixteen 
qc gc Li able options that iraprcre coding performance and provide encr res 
i 1 i ence . 

Video encoded at Ic* bit rates can become visibly distorted, especially 
those classified fitb high action, i.e., active met ion blocks. As roent 
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ioned above, the embed t men t S described herein refer tc the 11.263 staodar 
d. however aov suitable video ccdec standard can be employed with the em 
bodiments. Some of the characteristics cf the featues c( the IJ. 2 6 3 sta 
odard arc discissed below for iefcrmat i coa I purposes and are not meant t 
c limit the iDTCotici for a $ e with the H . 2 € 3 standard. One claracterisi 
ic cf the H.263 standard is the absence cf the group cf pictores (GOP) a 
fid higher layers in the 11 . 2 6 3 Standard. Where baseline encoded sequence 
s composed cf just a single i b t r a f r ame (I frame) fcilcwed bv a I ccf seqc 
eocc of interframes (P frames), the I c n g scqaence cf P frames provides g 
rciter compression ratios since the temporal redundancy it r eroc red bet'irc 
en consecutive frames. Howe t e r , motion e s. t i ma t i c d/ mc t i c a compensation 
(HE/MC) also creates a tempera! dependency snch thai errors generated dc 
ring the lossy ceding process will acenrcu late doring the decoding proccs 
s. The lack cf I frames prevents the decoder from breaking this accumtl 
ation of errors. The H.263 standard has a forced npdate mechanism such 
that the encoder must encode a roacrcblocl as an iotrablcck at least once 
ever? 132 times during the eoccding process. Figure 4 is a graph illus 
tratiug the effectiveness of the forced update mechanism. As illustrate 
d in Figure 4, the PS-MS cf the r j dec fluctuates randomly bet does cot dr 
ifl in any cue direction for frames later in the sequence. 
Figure 5 is a schematic diagram i 1 1 vsi rai i nf the determination of half 
pixel values in the II. 2 6 3 standard. As is well known, the 11.2 6 3 standar 
d uses half pixel interpolation for motion compensation. In the Jtatdar 
d, half pixel interpolation is indicated by metier eectcrs fit! 0.5 resc 
lotica (i.e. <7 . 5 , i.5». The encoder can specify in ierpolitico in the 
horizontal direction caly, vertical direction only, or both horizontal a 
nd Tertical directions. As illustrated by Figure 5. half pixel ra.foes a 
re found by bilinear interpolation cf integer pixel positions s d j r y o n d i n 
g the half pixel position. Pixel position .4 15 0-1. pixel position B 150 



(6 3) )03-348598 (P2003-(98 



-2, pixel position C 150-3, tod pixel position D 150-4, represent intege 
i pixel positions, while position c 152-1. position f 152-2, and positio 
d g 152-3 represent half pixel positions. Interpolations in the hcrixco 
tal direction may be represented as e = (A-B J -l)»l and i i t e r pc I a t i c n s in I 
he fcrtical direction may be represented as f = (A-C-j)»l . f n t e rpc I a t i cn 
s in the horizontal and vertical direction? ma? be represented as g=(AfB 
rCtD-Z)»2 

Figures <3A acd 66 are schematic diagrams of a baseline spatial video de 
coder and a compressed domain video decoder, respectively. The block di 
agram cf Fignre 6R rearranges some cf the fsncticoal blocks cf the spati 
a) denain video decoder cf Figurt 6.4. In particular, SLD 132 ana I DC T 1 
U are moved after MC 128 feedback loop. This arrangement keeps tbe iat 
ernal representation cf the videc in the compressed domain. The arrange 
ment of Figire 6B allows fcr the insertion cf compressed domain post pr 
ccessing modnies right after MC 128 feedback lecp. It s net Id be appreci 
ated that certain videc raanipe ! at ions . i o c Ti as cempes i ting, scaling, and 

deblocking, tc name a fey, are fasier in the compressed domain over the 
ir spatial demain counterparts . However, f r cm tbe videc codec pcint cf 
vie*, a spatial encoder is not perfectly matched tc a compressed domain 
decoder, As shewn in Figure 6B. tbe compressed domain video decoder dif 
fcrs from that cf the spatial domain video decoder cf Fignre 6A at sever 
a! points along the decoding pipeline. More than jest a rearrangement c 
f blccks, the points cf difference represent nonlinear operations, such 
as clipping and rounding. These points of ncnlinearity generate videc v 
1 1 fa differing PiiNK measurements between the two domains. 
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The nonlinear points are labeled as (i) T (ii), 
(iv) t and (v). In the spatial decoder of 
Figure 6A, IDCT block 134 transforms the 
incoming 8x8 block from the frequency domain 
to the spatial domain. The spatial domain 
values represent either pixel values or 
prediction error values for the color channels 
(Y,Cr,Cb). At point (i) of Figure 6A, the 
spatial values are clipped to the 

range (-255 £ x £ 256) . Note that there is no 

equivalent clipping operation at this stage for 
the DCT coefficients in Figure 6B. The 
second point of difference occurs during 
motion compensation. MC block 128 in Figure 
6A returns the pixel values from MEM 130 
referenced by the current motion vector. At 
point (ii) of Figure 6A, half-pixel (HP) 
interpolation 160, if specified, averages the 
neighboring pixel values and rounds the result 
to the nearest positive integer. At point (iv) 
of Figure 6B, half-pixel (HP) interpolation 
160 operates directly on DCT coefficients and 
rounds the result to the nearest positive or 
negative integer. Another point of difference 
occurs after the addition of the prediction 
error to the prediction value. At point (iii) 
of Figure 6A, the sum represents pixel values, 
which are clipped at block 162b to the 
range (0^x^255). Note that in Figure 6B similar 
clipping of pixel values is moved from the 
motion compensation feedback loop to the last 
stage of the decoding pipeline at block 162 
(point v ) . 

One skilled io the art vi J I appreciate that, MEM 130 is a frame buffer 
that stores the previous frame fcr mctico compensation. For the spatial 
demaic decoder, the frame buffer allocates eocuO memorf tc store the 



4 



(6 5))03-348598(P2003-98 



(Y,Cr,Cb) valaes f o i the incoming f i a me size. Fcr eumple, CIF videc sa 
rap I € d at 4:2:0 requires about 2 0 0 kilobytes cl memory. As MEM 130 is lb 
e single greatest source of memory usage in the videc decoder, a bjbrid 
data structure and inverse motion compensation methods defined herein al 
law fcr the reduction cf MEM usage fcr a compressed dcinaitj decoding pipe 
line, in cue embed imco t , two tc three times memory compression, without 

any significant less in the quality of the d c c c d c d videc, is achieved. 

Figure 7 is a block diagram illustrating the block transformations duri 
ng the video encoding and decoding process in accordance with one embed i 
roc iit cf the invention. The sequence of transformations above dotted lin 
e 170 describes the spatial compression methcdE used by the video encode 
r foi a block in an I-frame, cr a block in a P~frame after motion compens 
at ion/tnei icn estimation. Pixel block 172 is a full HI matrix. At this 

point, any compression cr truncation ic the spatial domain directly aff 
ccts the perceived quality cf the reconstructed block, After the DCT ir 
ansfcrm, however, transformed matrix 174 is compact v*ith the larger term 
s at lev? frequencies. The quantization step further compacts the block 
by reducing tc zero the smaller terms at high frequencies in block 176. 

The ligzag scan highlighted in block 176 orders the DCT coefficients fr 
cm low tc high frequency. The rnoicngth encoding discards the 2 c r o ccef 
ficients and represents only the nonzero DCT coefficients in a compact 1 
isi of Uc-valucd elements, e.g., run and level, in rnnlength represcnta 
ticn 178. Thns, memory compression in the DCT domain can be achieved by 

developing efficient data st lectures and methods that store and access 
rnnlengtb repiesentat iens cf the nenzerc DCT coefficients. 

fn one embodiment, a semi-compressed (5C) representat i en is cne such me 
mcry efficient funlengtli representation. Tbe ronleogth representation 0 
f the nonzero DCT coefficient; similar tc run length representations 178 
and 130 cf Figure 7. However, there are two modifications. Each tve-va 
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Iced element (run. level) is described by a cempcsite 16-bit value cf th 
e form: 

RL = binary r r n 1 1 1 1 1 1 1 1 1 1 i T (9) 

The 12 least significant bits (111111111111') define the value cf tbe d 
eioaat i led DCT coefficient from block 184, which were derived ficc qiaot 
ized block 182. it should be appreciated that block 184 is an example c 
f a DCT domain representation. It will be apparent tc one skilkd in th 
e art that tbe ralue cf the DCT coefficients can range from -2048 tc 204 
7. Block 186 of Figure 7 is a reconstructed block cf block 172 after an 
IDCT operation is performed cd blccl 184. Tbe four most s i gni f i can t- bi 
ts Cr r f r* } define the value cf the ran. Tbe ruo represents the position 
cf the ncozerc DCT coefficient relative tc the position cf the last ncn 
ic r c DCT coefficient according tc the tiftag scan in at 8x8 blcck. Sine 
e tbe run of a nonzero coefficient may exceed 15, an escape sequence is 
defined tc split the ran iotc smaller units. The escape seqeence RL=FC : 
is defined tc represent a mo cf 15 zero coefficients followed by a cce 
f f i c i e a t of zero amp ! i t o d e . 

In order to reduce the memory requirements, data structures tc store an 
d access the SC representation must be developed. The following data si 
roctoies were considered: array, linked list, vecicr. and hybrid. In de 
velcping these structures, a balance between tbe need fcr menicry compres 
sion and the need to maintain low ccrapotaticoai complexity is taken into 
ccflsi deration a a d discussed further with reference tc Table 1 belcw. W 
hile the SC representation provides tbe targeted memory compression, cer 
tain data structures will greatly increase tbe ccmpatat icaa I complexity 
cf tbe decoder in three areas. First, by emplcyicg the twe-byte represe 
otation, the values cf the (run, level) are no; immediately available. 
Functions tc pack and unpack tbe bits are needed for every access and mc 
dificatico tc these values. Secondly, motion compensation is no* ccmpli 
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catcd by the compact innJengtfe representation. Thirdly, sort acd merge 
c p e r a t i o o s are o e e d e d t c add the predict i c o error to the prediction. 
Figure 8 is a schematic diagram illustrating the use of a separate inde 
i tc find the starting position of each 8x8 block in the ruolccgtb repr 
esentaticn. If a single list I90 : alsc referred tc as teeter, is ised t 
c store the run length r ep r es en t a 1 1 cn for all Sx8 blocks 192-1 through 19 
2-4 in a frame, then access to a particular OCT block daring motion c crop 
ensaticG requires a separate isdei tc lockup its start position, which c 
explicates the oicticn c cmpe o s a i i c a . 

Figures 9A and 9R illustrate the sort a o d merge operations needed to ad 
d the prediction errcr tc the prediction for as array-based data nricto 
re and a iist data structure., respectively. !c Figure 9A an array-based 

data strocture requires only the addition of values at correspond leg a r 
ray indices. Rove re r, the array based data structure dees not offer mem 
cry ccmpiessicn advantages. In Figure 9 B , a list (or vector) data stric 
ture jequires additional sort and merge operations. That is, the merge 
algorithm requires ioserticn and deletion (unctions, tbich can be very e 
xpeosive in terms of computational complexity for data structsres such a 
s vectors. Mere pa r t i c a I a r 1 y , if indices are equal theo the DCT coeffic 
ieots can be added or subtracted, e.g., (0,20) 4 (0,62 0)*(0 f 64(l) . DCT c 
cefficients are inserted if index io error precedes that in prediction, 
e.g., insert (0,-3). DCT coefficients are deleted if addition of DC! va 
loes equals 0, e.g., (1,13) - (4 . -1 3) = (l , 0) . 

Table 1 compares the memory compression ratios aod computational costs 
for various data struct ores. While array-based data structures incur no 

additional c cmpa t a t i c n a 1 costs besides the 64 additions needed for the 
prediction updates, an array of DCT coefficients provides do meoicr) comp 
rcssicn over the array cf pixels since each DCT coefficient Deeds t»e-bj 
tes instead of one for storage A Mated list or vector cf semi-compres 
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sed (SC) reprcseotat icn provides ap tc 2.5 tines memory ccmprcssioo our 

the a j r a y cl pixels. Hcvercr, neither schticn is optimal since the Id 

sert ion/deletion test for a vectci is expensive, especially insert igos a 

do deletions in the middle cf the v c c t c r and the raemcrj eve i head fcr a I 
inked list is expensive, as internal pointers are created fcr every elem 
cm i o the list, 

TABLE 1 



Data Structure 


Memory Size 
(kilobytes) 


Insertion/Deletion 
Cost 


Memory Overhead 


Compression Ratio 


Array of Pixels 


152 


None 


None 


None 


Array of DCT 


304 


None 


"None 


None 


Vector of SC 


60 


Expensive 


Minimal 


2.5:1 


Linked List of SC 


60+overhcad 


Moderate 


Expensive 


2.5:l(w/o overhead) 


Hybrid of SC 


70 


Moderate 


Minimal 


2.2:1 



A hybrid data structure fcr the SC representation provides the optimum 
balancing cf the competing interests of Tabic 1. The hybrid data struct 
nre is developed tc take advantage cf the Ic* ccmpui at icnal cost cf the 
array structure cf Figure 9 A and the high ccmpressicn ratio cf the vecto 
r structure cf Figure 9 B . The hybrid data struct tire consists cf 3 fiied 
-siae array that holds a fixed n amber of DCT coefficients per block and 
a variable-size overflew vector that stores the DCT coefficients cf thes 
e blocks that exceed the fixed size array alloc a Lion. It should be appr 
eciated that the fixed size array can be configured tc hold any s g i table 

number cf DCT coefficients per block, wherein the cumber of DCT cceffic 
ients is less than 6<1, Of course, as the fixed size array becomes great 
er the amount cf memory c cmp r e s s i on is decreased. Id one embodiment, th 
e fixed me array is configured to hold 8 DCT coefficients per block. 

Figure 10 is a schematic diagram of a hybrid data structure including a 
n array structure and a rector structure to alio* for memcrj ccinpressi cn 

and computational efficiency in accordance with one embodiment cf the i 
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nfenticn. DCT blocks 2 0 0 - 1 , 200-2 aod 200-n include z c x o DCT coefficien 
ts and non-zero DCT coefficients. It should be appreciated thai DCT bio 
cks 2 0 0 - 1 through 2 0 0 - n represent the DCT domain representation as discs 
ssed above with reference tc Figure 2. in addition, blocks 200-1 thrcug 
h 200-q are associated with blocks of a frame cf video data, e.g., block 
184 cf Figure I. The non-zero DCT coefficients for each cf blocks 200- 
l through 200-q are identified and inserted into fixed size array 202 da 
t a structsre. Fixed siie array 2 02 includes filed she biccks 204-1 thr 
OEgh 2 0 4-c. In one embodiment, each block 2 04-1 through 2 0 4 - a is sized 
tc store 8 DCT coefficients in ac 8x1 data structure. It should be appr 
eciated that, the invention is net limited tc biccks configured tc store 
8 DCT coefficients as any suitable siie may be asea. As stated abous i 
s the capacity of the blocks increases the amenst cf memory compression 
decreases. 

Still referring tc Figure 10 t where there are mere than 8 non-xerc ccef 
ficieots in any cf DCT biccks 20O-1 thrcugh 200-n, the non-zero DCT ccef 
ficienis exceeding the capacity cf respective fixed size blocks 204-1 t h 
rough 204-o are placed in overflew vector 206. Overflew vector 206 is c 
onligured as a variable siie overflow vector, i.e. the overflew vcctci t 
s dynamic. Fcr eiample, block 200-1 includes 9 non-iero DCT coefficient 
s Al-AJ. Here. DCT coefficients Al-AS are copied tc filed siie block 20 
4-1, while DCT coefficient A9 is copied to cverflc? vector 206. Block 2 
00-2 incUdes 10 non-zero DCT coefficients Bl-BlO. Here, DCT ccefficien 
ts Bl-BS are copied tc filed siie blcck 204-2, while DCT coefficients B9 

and BIG tie copied tc overflew vector 206 and so on fcr each blcck c! t 
he frame. Index tabic 205 contains entries which identify cor r espendi Qg 

fixed sue biccks 204-1 through 204-n fcr the entries in cverflc? vectc 
r 2 0 6. The siie cf the index table is negligible as each entry is 1 by t 
e. Accordingly, fcr a irame cf data corresponding to DCT blocks 200-1 t 
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hreigh 2 0 0 - d f data from fixed size array 2 0 2 a d d cvcrflc* vcctcr 2 0 6 are 
combined tc product image 210. It should be appreciated that the s a v i d 
gs in memory is substantial. That is, DCT block* 2 0 0 - 1 tbrccgb 2 0 0 - n ai 
e reduced ficm 64 zero and ocn-zerc coefficients to 8 ocn-zerc cccfiicic 
ats, or less, stored in fixed sire blocks 2 0 4- 1 through 2 0 4 -n io reesi in 
stances- 01 cccrse, mere cr less ncn-zerc coefficients may be provided, 
wherein the non-zero coefficients in excess cf 8 are stored in overflew 
vector 206. 

Figures 1 1 A through 1 1 C are graphs illustrating the factors evaluated i 
a determining the capacity cf the fixed size blocks of the fixed siiear 
ray and the overflow rector of the hybrid data structure in accordance w 
ith one embodiment cf the invent ion. In Figure 11 A. the average onraber 
cf non-zero DCT coefficients per luminance block for two typical C1F seq 
uences is depicted by lines 220 and 222. The nnmbcr cf ccn-zcrc DCT ccc 
fficieots per block ranges from three tc seven. That is. cf the 64 cccf 
fieients, coly 2-7 coefficients are ncn-zerc coefficients co average. V 
sing the information from Figure 1 1 A as a gnide ; Figere 1 1 B illustrates 
that as the fixed-size array increases, the size cf the overflew vector 
decreases, thereby minimizing the insertion and deletion ccsts cf the ve 
ctcr. Here line 220-1 corresponds tc the CIF sequence cf line 220 cf Fi 
gure 11 A, while line 222-1 corresponds tc the CIF sequence cf line 222 c 
f Figure 11 A. One skilled in the art will appreciate that as the fixed 
siie array increases io terms of capacity, the memory compression decrea 
ses. Additionally, Figure 11C illustrates that the lead factcr cf the a 
rray decreases as sell, indicating that much cf the array remains empty. 

In one embodiment, a fixed-size array that holds 8 DCT coefficients pe 
r blcck is chescn. Here again, line 220-2 corresponds tc the CIF seqnen 
ce cf line 2 2 0 cf Figure 11.4, while line 222-2 corresponds tc the CIF se 
qneoce cf line 222 cf Figure 11 A. This choice minimizes the size cf the 
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cverflcv vector tc about 200 DCT coefficients and maintains a lead fact 
or of between a b c h t 95c aod abcet 155c. It will be apparent tc one skille 
d is the art that tte size cf the fixed array is net limited tc 8 coeffi 
cients per block and that a l y suitable number cf coefficients per blcck 
may be chosen. Ad d j t i c oa i i y , the i o d i t i d u a I blocks cf the fixed size ar 
i a j say bare any suitable cc n f i ga i a t i c o . Fcr example, a blcck capable c 
f holding S coefficients say be arranged as an 8 x I blcck, a 4 s 2 bice 
fc, etc., while a blcct capable cf holding 9 coefficients may be arranged 

as a 9 x 1 blcck, 3 x 3 blcck. etc 

Figure 12 is a flowchart cf the methed operations fcr reducing tie memo 
rv requirements for decoding a bit stream, in accordance with cue embed i m 
ent cf tbe invention. The met bed initiates with operation 230 where 2 v 
idee bit stream is received, h one embodiment, the bit stream is a low 

rate bit stream. Fcr example, tbe if I dec stream may be associated with 
a video coding standard Such as H.263, Mctioo Pictures Expert Group (MPE 
G- 1/2/4) , H.261, Jcint Photographic Expert Crcup OPEC) , etc The mclhc 
d then proceeds to opera t icq 232 where tbc frame of the bit stream is de 
ceded iotc a discrete cosine transform (DCT) domain representation fcr e 
acb blcck cf data associated with tbe frame Here, the video is process 
ed through the first t?o stages cf a decoder, such as the dcccdcr cf Fig 
tires 2. 6B and 15. That is. the video data is processed through the var 
table iength decoder stage acd the dcqnaotisaticn stage tc decode the co 
mpressed bit stream into a DCT domain representation, It should be appi 
eciated that the DCT domain representation is in a compressed state form 
at. Tbe frame is decoded cae block at a time. The method then moves tc 
opcraticn 234 where tbe non-zero coefficients cf the DCT domain represe 
ntation are identified. Here, est of the 64 DCT coefficients associated 
with tbe DCT domain representation for a block of data, relatively few 
cf the U DCT coefficients are typically non-zero coefficients. 
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Still referring tbe Figure 12 ; tbe method then mcves U operation 236 w 
here a hybrid data stroctire is assembled. The hybrid data siractare is 
eludes a fixed sire array a o d a variable me overflew vector. One exern 
p 1 a r y hybrid data structure is the fixed size array that includes a p 1 u r 
aiity cf fixed size block and the variable size overlie* vector with rei 
erence tc Figure 1 0 . The method then proceeds tc operation 2 3 8 where ih 
e non-zero coefficients cf tbe DCT domain representa t i cn arc inserted in 
tc the hybrid data structure. As mentioned fitli reference tc Figure 10. 

the o c d - 2 € r c coefficients for a DCT domain repre sen t a t ico fcr a block c 
1 video data are associated with a fixed size block in the fixed size ai 
ray. If the number cf non-zero coefficients exceeds the capacity cf the 

fixed size block associated with the block cf video data, then the rema 
ioing n c d - 2 1 1 c coefficients are stored io the variable size cverflcw vec 
tor. (n c d e cubed imcn t , an index table maps the data in the overflow ve 
ctor back to tbe appropriate fixed size block in the fixed size array. 
Thug, the memory requirements are reacted thrcngh the hybrid data struct 
nre and the storage cf the non-zero coefficients. Mere particularly, th 
e memory requirements cao be reduced by 56 r rt ^ithcii any loss of video qu 
a I i t y , 

It should be appreciated that the oco-jerc coefficients fcr each DCT dc 
main representation associated frith a frame cf data are stored io ike hy 
brid data structure. The stored data fcr the frame is then combined and 

decompressed for display. Once tbe next frame is decoded into a DCT dc 
main representation tc be stored io the hybrid data structure, the data 
in the hybrid data structure associated with the previous frame is flush 
H , in one embediment. As will be explained further belctv, inverse mcti 
cn compensation is performed od the stored data in the compressed domain 
. The inverse met icq compensation uses integer approximation Icr full p 
ixel ieverse motion c cmp e o s a i i c a and factorization for half pixel infers 
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c motion c ompe q s a t ion . 

The mi i d components in the spatial H.263 ridec decoder include runlecgt 
h decoding, inverse DCT T and i o v e r s c motion compensa t i c n . Us i o g a tiraio 
I profiler, the performance cf TELENOR' S H.263 video decoder cg a 1.1 GH 
t Pentium 4 prccessoi is measured for baseline data. Decoding a baselio 
e video and ignoring system. calls, the profiler measures the cural! tiro 
e it tikes to decode 144 frames and details the timing characteristics c 
[ each component. Table 2 is a timing profile for the spatial H.263 via 
ec decoder and highlights the timing results for select fnncticBS. 

Table 2 



Function 


Function Time (rns) 


Hit Count 


Picture Display 


772 


144 


Inverse Motion 
Compensation 


243 


56336 


RunJengtb Decoding 


57 


39830 


Inverse DCT 


3 


42253 



Table 3 is timing profile for the ncn-cpt imiied compressed d cms in H.263 
video decoder. Oct exemplary decoder pipeline configuration is the dec 
cder with reference to Figure 2. 



Table 3 



Function 


Function Time (ms) 


Hit Count 


Inverse Motion 
Compensation 


9194 


56336 


Picture Display 


1547 


144 


Runlength Decoding 


32 


39830 


Inverse DCT 


652 


340197 



As shewn in Table 2. the spatial domain video decoder takes about 1.2 s 

eccnds tc decode 144 frames. The majority cf the time is spent in the P 
ietareDispIay function, which converts the ecler tallies cf each frame 

from TBV to RGB in order to display it gc a suitable operating system, s 
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cch as WINDOWS 1 * 11 . Functicos s a c h as ronlengtb decoding, inverse DCT, an 

d inverse actios ccmpensatiCD take abcat 25% cf the tclal time required 

to decode the video. Inverse mctico ccrapens at icn is especially fast in 

the spatial domain. Here, fall pixel met icn compensation simply sets a 

pointer to a pesitica i n lienor r cr a frame buffer and copies a block cf 

data, while half pixel mctico ccmperisa t ice sets a pointer in mercery and 

interpolates values asing the shift operator. In ccatrast. Table 3 high 

lights seme cf the timing results for a non-optimized compressed domain 

video decoder. The nco-cpt imi zed compressed d c ma i d decoder takes about 

13.6? seconds tc decode the same M4 frames. 
The main bottleneck for the compressed domain decoder is the 

inverse motion compensation function. As described in Eq (8) 
above, full-pixel inverse motion compensation in the 
compressed domain requires a sum of four (ZAf,) terms, where 
JM t is defined as pre- and post- multiplying the 8x8 matrix 
block FVwith transform matrix C 9 . 

F k = m x +TM 2 +TMi+TM A (10) 
where 2M, = QF'.Q (11) 

Table 4 defines the full-pixel transform matrices Here, S 

represent the 8x8 DCT matrices, and Uk and Lv are defined in 
Equations 3-6 above. 



Table 4 



Full-pixel transform matrix 


Matrix definition 


C -C 


sujr 


c =c 




c =c 


SLJ' 


^22 ^42 
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Each 8 x S matrix multiplication require! M'l multiplies and 4 4 S aaditicn 
s. As is iccwD matrix mo 1 1 i pi i ca t ion if ccmpu ta t i cqj 1 1 y expensive. Tib 
le 5 c cmpa j e s the cpt imi zat i co s c hemes , soch as matrix appro* i ma i ico . ca 
trix factorization, e h a redbl cck for raacublccks. and a hybrid scheme fcr 

a ccmpiessed demain video pipeline seek as the pipeline with reference 
tc Figures 2, 6B aod 15. The compressed demain videc decoding pipeline 
sbcnld deccde ai a rate cf abcat 15 25 frames per second (fps) in order 
tc provide acceptable qealit? fcr handheld devices that sippen video fc 
rmaU such a* the cenraoc intermediate fcrmat where each frame of data co 
ntaiis 352 liaef with 2 S 8 piiels per lioe. 



Table 5 



Optimization 


Decode Time (s) 


# Frames 


FPS 


Comments 


Spatial domain 


9.79 


144 


14.71 


Original TELENOR H.263 
video decoder. 


Matrix-matrix 


14.17 


144 


10.16 


Full 8x8 matrix multiplications 
for TM 


Approximation 


9.82 


144 


14.66 


Good time but poor PSNR. 


Factorization 


12.95 


144 


11.12 


Good PSNR but poor time. 


Sharedblock 


14.85 


144 


9.70 


No improvement here. 


Hybrid 


9.83 


144 


14.65 


Good time and good PSNR. 
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One enhancement to a compressed domain video decoding 
pipeline is to reduce the number of /^operations in Eq (10) by 

block alignment. For example, to decode 144 frames of a 
sequence and measure block alignment rates at 36.7% of all 
blocks. Figure 13 is a schematic diagram illustrating three 
examples of block alignment to reduce matrix multiplication. 
Block alignment case 240 where (*> = 8,/i = 4) , block alignment 
case 242 where (*f = 4,/» = 8) 7 and block alignment case 244 where 
(H> = 8,/r = 8)are each illustrated. In each of these examples 240, 
242, and 244, 2M ( operations are eliminated when the overlap 
with a corresponding block is zero. However, it should be 
appreciated that, in the DCT domain (compressed domain), 
block alignment does not yield savings when half-pixel 
interpolation is specified. The equations for half-pixel inverse 
motion compensation in the compressed domain are given below. 
For the example of (w = 8,ft = 8) , half-pixel interpolation still 
requires four Th4 i operations as illustrated in equations 12 and 
13. Table 6 is provided for informational purposes to define 
the half pixel transform matrices Chpij. 

TM^CvfCw (13) 



Table 6 



Half-pixel transform 
matrix 


Horizontal mtcrpoistjon 


Vertical interpolation 


Horizontal & vertical 




SU„S' 






C -C 


SL_ t !T 




+Z^,)S' 


' C ~C 




sljt 













• t 
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It should be noted that even fci a perfect ly aligned DCT blcck, half-pi 
xel iotcipclai ico creates an ever lap cf one with the neighboring blccks. 

Figure H is a schematic diagram cf a bail pixel i nierpcl a t ico fci a p 
erfectl? aligned DCT block. The half pixel i c teipc 1 a t ico create! c?erla 
p p i n g into neighboring blccks by c o e pixel width aod one pixel height. 

Increasing the speed cf process i n f in the compressed d c ma i 11 decoding pi 
pelinc mar be accomplished by rearrangement cf the functional blccks cf 
the decoder cf Figs re 2. With reference tc Tables 2 and 3, the process i 
Dg time for the inverse DCT block is much less is the spatial demaic (3 
ras) than in the compressed domain (69? ms) . In the spatial domain, icve 
isc DCT is applied before tbc feedback Iccp to the iotrablccks and the e 
rrcr coefficients. Id particular, the i at r a b locks and error coefficient 
s make op less than ISX cf all the blocks id the video. The ether 11% 0 

ft 

1 the time the ioversc DCT function is simplv skipped, tn the ccmpresse 
d demaio. inverse DCT is applied at the last stage of the pipeline tc 10 
0 c c cf the blccks in each frame of tbe video. 

Figore 15 is a schematic diagram illustrating tbe rearrangement of the 
functional blccks cf a compressed domain video decoder to enhance the pr 
ccessing cf the video data in accordance with cne embodiment cf the inve 
otion. Here, the functional blccks are rearranged and the compressed dc 
main pipeline is split at two points. The first split occurs after VLD 
124 and DQ 126 at point !i) 252. ic the upper branch, the pipeline keep 
s an internal DCT domain representation for memory compression 128. In 
the lower branch, the pipeline moves the RLE and IDCT ap tc the frcnt tc 

decode tbe error coefficients into the spatial domain. The second spli 
t occurs during motion compensation. CMC ) at point (ii) 2 5 4. During mcti 
on compensation, a spatial domain cutput may be generated according tc e 
qoatici (7). The cutpot can be directly added to the error coefficients 
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tc reconstruct the current block at pcint (iii) 256 tc be presented en 
display 136. DCT block 250 t< inserted in the feedback loop tc maintain 

the internal DCT r e p r e $ e d r a t ico . The combination cf R LD 132 and I DC T 1 
34 at point (i) 252 and the DCT at pciot (ii) 254 requires less computet 
ion than the I DCT block at the 'last stage cf the pipeline in Figure 2. 
Table 1 show* that the rearrangement with reference ic Figuie 15 generat 
es a 2 OS speedup tbat can be combined in addition tc ether c.pt train iicn 
schemes described herein. 

Table 7 



Function 


Percentage of Blocks 


Comments 


IDCT in Figure 15 
point (i) 


15% 


Intrablocks and error coefficients * 
represent small fraction of all blocks. 


DCT in Figure 15 
point (h) 


63% 


Non-aligned blocks require DCT, but 
aligned blocks are directly copied 
without DCT. 


IDCT in Figure 2 


100% 


Applied to all blocks in DCT domain. 
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In one embodiment, the inverse motion compensation is 
accelerated by reducing the number of multiplies required by 
the basic TM operation in Eqs. (11,13). Instead of calculating 
full 8x8 matrix multiplications, the DCT matrix S is factored 
into a sequence of sparse matrices as illustrated in Eq. 14. The 
sparse matrices in Eq. (17) include permutation matrices 
{A i,A2>A3 t A4*A $,Ae) and diagonal matrices (D 9 M). Substituting 
this factorization into Eq. (15), we derive a fully factored 
expression for TM t in Eq. (16), which requires less multiplies 
than the original Eqs. (11,13). 

S=D4A t AJM t A s A e (14) 
TM, = Sc n S'F,Sc a S' (15) 

(16) 









- 




1 




i 


D = 


\ o" 

** 


Ai = 


i 

1 




1 


A 3 = 


1 0 

1 1 

4 1 




V 

0 st 




l 

i 




l i 




o \'. 






1 




0 i i 

1 -t 








fa 




1 

_ — 


* 


.1 t 




J* 












1 1 






M = 


* 

41 -T- 
* 


A 4 = 


i-i o 

t 1 
t 

1 

0 i 




* * 0 
1 .1 

1 -1 
0 i • 


A,= 


1 I 




1 




1 

1 


* 


f 1 
I 




1 o A 



(17) 



D = diag{0.3536,0.2549,0.2706,0.3007,0.3536,0.4500,0 6533, 1 .28 14} (18) 
A = 0.7071, 5=0.9239, C=0.3827 (19) 

Thus, the mtrix multiplication is replaced with matrix perrnutat ice. H 
c w e v e r , a full? hctcred express i co fct tie term TMj , as shewn in Eq. (1 
6), docs not necessarily speed up inverse motion compensa t inn. lo esstn 



(flO) )03-348598 (P2003-CB98 



ce, multiplies have been traded fci i&ecci) accesses, and too man? memory 
accesses can actually sic? down the decoding process. Therefore, Ike m 
atiices are regrcsped tc strike a balance between these coupelinf fundi 
cnalities. Matrix S (= G qG j ) is factcred intc two terras: GjpDAi^j, mi 
xture ci permutations and mu 1 1 i p i i ca t i ocs ; tod G^MAjAgAg, mixture of pe 
r (notations and additions. The fixed matrices j , Kj are defined and sub 
stttated into Eos. (10 and 12) tc form a factored expression for ioTerse 
rocticn ccmpensat i c n in Ec. (20' 
J h = cjj[ , J w = G x c n = G^ 

Similarly for hall-pixel inlerpclaiion: 

Kh ~ C h&\ G \ - C kp4\ G \ » - G \ C hp11 - G \ C hpZZ 

+ K h G t 0 F , 3 G 0 J w + K&F* 4 G 0 KJS' 



(20) 
(21) 



(22) 
(23) 

(24) 



Further speed enhancement may be obtained by implementing fast muitipli 
cation by the fixed matrices Jj, Kj. The fixed matrices contain repcatc 
d structures. For example, the matrix Jg is defined as fellers 



1 


-1 


- a 


0 


b 


a 


c 


0 


1 


1 


- a 


-1 


b 


0 


c 


0 


1 


1 


- a 


-1 


-b 


0 


— c 


0 


1 


- 1 


- a 


0 


-b 


- a 


- c 


0 


1 


-1 


a 


0 


c 


- a 


-b 


0 


1 


1 


a 


1 


c 


0 


-b 


-1 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 



where a = € . 7 0 7 1 , b= 0 . 9 2 3 9 1 and c = 0 . 38 2 7. To corapBie u = J 6 *, wbcre u= 
p...n<jl and f= • v j , . , , Vg| , a sequence cf equal i cos is calculated accord i 
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og to the foliCKing steps: 




•V. = V . +V 2 


(25) 




(26) 


y 3 =av 3 


(27) 


y* =av « 


(28) 


y>=y-y 3 


(29) 


y 6 =y$-y4 


(30) 


yi-yi -y A 


(31) 


y^y^y* 


(32) 




(33) 


y t0 = 


(34) 


^11=^7 


(35) 


yn = y*~ y^o ~ . v h 


(36) 


y» = yw> ~yu 


(37) 




(38) 


"i=y*+yn 


(39) 


»j =y 6 -y a 


(40) 


«4=y2~y,-y n 


(41) 


u s=y2 + y7 + y i3 


(42) 




(43) 


a, = 0 


(44) 


», = 0 


(45) 
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Accordingly, the matrix- vector multiplication has been 
transformed into a sequence of equations. The above sequence 
of equations requires 5 multiplications and 21 additions. The 
matrix multiplication Jftf * n (24) requires 104 

multiplications and 164 additions. Thus, a S time reduction 
oyer the number of multiplies needed for matrix 
multiplication C rt F'in Eq. (8) is achieved here. Additionally, no 

precision is lost during this matrix operation, which uses 32- 
bit floating point arithmetic. However, with reference to Table 
5, factorization speeds up the compressed domain pipeline by 
only 9% over matrix-matrix. Consequently, the extra memory 
accesses slow the frame rate to below the target rate of about 

■ 

15 to about 25 fps so that factorization alone will not suffice. 

To further speedup the inverse motion compensation the 

multiplies required by the basic TM operation in Eqs. (11,13) 
are eliminated. The full-pixei and half-pixel matrices C y and 
C hpi j are approximated to binary numbers to the nearest power of 

2 5 . By approximating these matrices with binary numbers, 
matrix multiplication can be performed by using basic integer 
operations, such as right-shift and add, to solve inverse motion 
compensation in Eqs. (10,12). For example, the full-pixel 
matrix C n where h«l is examined below. It should be 

appreciated that the other matrices are approximated in a 
similar fashion. 



0.12501651 -0.17338332 0.16332089 ■» -0.03447659 
0.17340244 - 0.24048958 0.22653259 ■» -0.047*2041 
016334190 -022653624 0.21338904 — -0.04504584 



(46) 



0.03449197 - 0.04783635 0.O45OS013 



-0.00951207 



• i 
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Where each elemeci in the matrix is founded to the nearest powers of 2 : 
matrix (47) results: 

(47) 



0.1250 -0.1875 0.1875 •- -0.0625 

0.1875 -05500 0.2500 •■- -0.0625 

0.1875 -02500 0.1875 - -0.0625 

* • « ■ * 

* ■ * • ■ 

0.0625 -0.0625 0.0625 0 



Since the DCT elements lie in the range of [-2048 to 2047], 
direct shifting of the DCT coefficients would drive most of the 
values to zero. In order to maintain precision in the 

intermediate results, we scale each DCT coefficient by 2* 
throughout the decoding pipeline. This scaling factor is 
introduced during the quantization and dequantization steps so 
that no extra operations are incurred. 

Furthermore, we implement fast matrix multiplication by 
grouping terms according to the sum of products rule (see Eqs. 
(48-50)). 



a, = 0.1250i/, -0. 1875* 2 + 0.1875v 3 -0.125<fc 4 + 0.1250>y-0.125(k 6 + 0.0625» 7 -0.0625v 8 (48) 

»3)-(v 2 »3)-(v 2 »4)+(v 3 »3)+fV, »4)-(v„ »3) + (v 3 »3)-(v 6 »3)+(v 7 »4)-<v, »4) 

(49) 

a, = (v t -v 2 + v 3 - v 4 +v 3 - v 6 ) »3+(-v, + v, -i-v 7 -v s ) » 4 (50) 

The computation for w = C n v, where «= {uj t .„,u#} and v={vj t ...,v s } 1 may be calculated 

as: 

"t = (v, - v ? + v 3 - v 4 + v 3 - v 6 ) »3+ (>v 2 + v 3 + v 7 - v n ) »4 (51) 
" 2 = (v a - v 2 ) » 2 + (v, - v 4 + v s - v t + v, ) » 3 + (v, - v 4 + v, - v R ) » 4 (52) 
"3 =< v i + v > ~ v 4 +v 3 -v 6 ) »3-(v 2 »2)+{v, -v 4 +v 5 +v T -v 8 )»4 (53) 
u * = ( v i " + v, - v 4 + - v s ) » 3 + (v 3 - v 2 - v 4 + v 7 - v 8 ) » 4 (54 ) 

"s = ( v i - v 2 + v 3 - v 4 + v 3 - v 6 ) » 3 + (-v 2 + v 3 + v 7 - v„) » 4 (55) 
u 6 = (v, - v 2 + v 3 - v 4 + v,) »3+ (v 7 - v 6 ) » 4 (56) 
*t = ( v i + v 3 - v 4 + v 3 - v 6 + v7) » 4+ (v 2 ) »3 (57) 
«k = (v, -v 2 + v 3 - v 4 + v 5 ) »4 (58) 
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The matrix approximation requires a total of 17 right-shifts 
and 57 adds. The matrix approximation QF'in Eq. (8) requires 
136 right-shifts and 456 adds. Accordingly, a significant 
reduction in complexity over matrix multiplication is achieved 
with floating point precision. In fact, Table 5 shows that 
approximation techniques speed up the compressed domain 
pipeline by 31%, which is enough to achieve the target frame 
rate of about 15 fps. However, the PSNR for a sample video 
decreases and shows noticeable drift in areas of moderate 
motion. 

A hybrid I ac I c r i za t i on/i n t e ge i appro* imajt i cn f g r t b e transform matrix T 
M (bat is selectively applied based open the videc motion provides the d 
e s i r e d frame rate of between abcat 15 and abect 25 fps, while maitiiioio 
g acceptable quality. As ment i cned a b c ? e . the integer approximation tec 
bniqoe reduces the complexity cf the decoder but a I s c reduces the PSNR c 
f the decoded videc. At the same time, the factor iiat ten method maintai 
ns geed PSNR but dees not reduce the complexity cf the decoder tc meet t 
he desired frame rale. Through the integration cf the lew complexity cf 

the integer appr cximat i c o with the high precision cf the facuriiatico 
method a compressed domain tidec decoding pipeline fcr seppcrting a low 
rate ridec bit stream is obtained. 
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Two types of transform matrices have been discussed herein: 
7M f , fuU pixel motion compensation illustrated in Eq. (11); and 
TM^, half pixel motion compensation illustrated in Eq. (13). 

Full pixel motion compensation, using approximate matrices 
for TM i , has only 28% of the computational complexity 

compared to that of using 8x8 floating point matrices. 
However, when applying the approximation techniques directly 
on the half pixel transform matrices, it has been observed 

that half pixel motion compensation, using approximate 
matrices for IM^, lowers the PSNR (see Table 8) and creates 

visible distortions in the decoded video. The errors are 
generated from two sources. First, the half pixel transform 
matrices 7M M are more sensitive to approximation techniques. 
With reference to Table 8, TM^ are composite matrices, 
composed of many more terms than Secondly, as described 

above with reference to Figures 6A and 6B, the nonlinear 
processing during half pixel interpolation, combined with the 
errors generated by the approximation techniques, lead to an 
accumulation of errors that are especially visible in regions of 
moderate to high motion. 



• * 
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The selective application of the factorization method to the 
half pixel matrices addresses these errors. As discussed above, 
the factorization method maintains floating point precision so 
that the errors described can be minimized. For example, the 
factorization method reduces the matrix multiplication with 
TM^ into a sequence of equations similar to those described in 

Eqs. (25-45). These equations maintain 32-bit floating point 
precision so that no approximation errors are generated. 
Furthermore, the factorization methods decode the DCT block 
into the spatial domain during motion compensation so that the 
optimizations described with reference to Figure 15 may be 
combined with those described here. Table 5 shows that the 
hybrid method meets our target frame rate of 15 fps, while 
Table 8 illustrates that the PNSR of the hybrid method provides 
an acceptable PSNR. 



Table 8 



Video 

(128 kbps, QCIF, 
15fps) 


Compressed 
Domain w/ Factor 
TM(PSNR Y) 


Compressed Domain 
w/ Hybrid TM 
(PSNR Y) 


Compressed Domain 
w/ Approximate TM 
(PSNR Y) 


Sample A 


25.53 


25.53 


22.65 


Sample 8 


22.47 


19.57 


18.75 


Sample C 


30.79 


30.66 


29.90 


Sample D 


33.29 


33.25 


28.93 


Sample E 


31.27 


31.10 


28.89 



Figoie 16 is a flowchart diagram cf the method operations fcr perforata 
g inverse motion c cmpen s a t i co id tie compressed domain in accordance *i t 
h one embodiment of t It e iertatififl. T b e melted initiates *ltb cperaticn 
2 6 0 where a frame cf videc data within a compressed bit stream is receiv 
ed. In cne embodiment, the bit stream is a lew rate bit stream, fcr ex 
ample, the bit stream may be associated \ritb a known video ceding S t a o d a 
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rd, such as MPEG 4, H.263. H.26I, etc. The method then advances tc cper 
at ion 262 where a block of the frame cf the bit site am is decoded into a 

discrete cosine transform (OCT) domain represe d t a t i en. Here, the video 

i$ processed through tbe first twe stages cf a decoder soch as. the decc 
der of Figures 2, 6B and 15. Hat is. the video data is processed thrcn 
gb the variable length decoder stage and tbe deqvacttzaticn stage tc dec 
cde the compressed bit stream into a DCT domain representation, ft shci 
Id be appreciated that the OCT domain representation is in a compressed 
State format. The method then proceeds to operation 264 There the data 
associated with the DCT domain i epieseo 1 at icn is stored in a hybrid data 

structure. S suitable hybrid data structure is the hybrid data stracto 
re discassed with reference tc Figures 10 and 12. In one embodiment, th 
e hybrid data structure reduces the memory requirements for a portable e 
lectrcoic d e f i c e ; e.g.. cellular phone, PDA, web tablet, packet personal 

computer, etc., having a display screen for presenting the video data. 



Still referring tc Figure 16. the method moves tc operation 2 6 6 where, i 
nverse motion compensation is performed on the data associated with the 
DCT domain representation in the compressed domain. Here, tbe inverse m 
c t icn compensation includes selectively applying a hybrid factor ixai icn/ 
integer approximation technique described abeve with reference tc Tables 

5 and 8. The method then advances to decisicc operation 268 where the 
hybrid f ac tor i:at ica/i nteger a p p r ex i ma t i o n identifies a type cf transfer 
m matrix associated with the block cf video data being processed. In on 
e embodiment, the type of transform matrix is detected through infermati 
on in i bit set cf the bit stream being decoded. If the transform matii 
i is a half-pixel matrix then the method proceeds to operation 2?0 where 

a factorization technique is applied ic decode Hie bit stream. In one 
embodiment, the factorization technique redaces matrix ma 1 1 i pi ica t i cn in 
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tc a scries c f equations as described a b c t e ? i t h reference tc eqeaticn 2 
5-45. That is, matrix multiplication is replaced with matrix pcrmuiatic 
d. If the transform matrix is determined tc be a foil pixel matrix in d 
ccisico operation 268, then the uethod advances tc operation 272 vhere a 
a integer approximation technique is applied ic deccde the bit stream. 
Here, the matrix rnu 1 1 i p 1 i ca t i on may be perfcrined fcy using basic integer 
operations tc solve inters* met icq compensation as discussed above with 
reference tc equations 46-58. Thus, thrcagh the selective application o 
f the hybrid factcrinticn/ integer appinximaticn technique, pjeccssing i 
a tbe compressed domain is performed tc prcriric a snfficient frame rate 
villi acceptable quality tc enable the reduction in memory achieved three 
the hybrid data structure discussed above . 
Figure 17 is a schematic diagram cl the selective application cf the hy 
brid factcr iiat ice/integer apprcximaticn techniqne in accordance vitfa cn 
e embodiment of tbe invention. Display screen 280 is configured tc pres 
cut images defined by lev bit rate lidec. For example, display screen 2 
80 may be associated *ith a portable electronic device e.g., a PDA, cell 
clar phone, pecket personal computer, *eb tablet t etc. Ball 2 8 2 is mcvi 
ng in a vertical directicn in the video. Blocks 284 are located around 
tbe perimeter cf the moving object and are considered high or moderate to 
ction areas and change from frame tc frame. Blocks 2U represent the ba 
ckgrcnnd and remain substantially the same from frame tc frame. Thus, d 
iring the decoding cf the ecmpressed bit siream blocks 284 cf a frame of 
data will be associated with high rncticn areas, from frame tc frame, vb 
ile blocks 586 remain substantially the Same from frame to frame. Blcck 
s 2S4 which are associated with the high rncticn areas, require higher pi 
ecisicn during decoding techniques. i.e., factorization, vbife blocks 28 
6 remain substantially the same and can tolerate a lower complexity inte 
rpclaticn method, i.e., integer approximation. Therefore, the factcriia 
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ttCD techniqie is applied tc the high and moderate nctioc area blccks U 
4 and tbe ioteger apprciinat icq is applied to backgrcnnd blccks 2S6. As 

meot icQed above, infcrmat ico embedded in the bit stream is detected to 
determine whether a block is associated ffith high motion, i.e., half pix 
el mctico ccmpensaticn is applied tfircoflh f a c t or i 1 a t i en , or if the block 

is backgrcend data, i.e.. foil pixel DCticn c crcpe n g a t i c n is applied tkr 
otgb integer a p p r ox t ma t i c o . Io cae embed imci t . tbe mcticn vectors »iU 
reference tc Figures 2, 6B t and 15 specify whether the met Icq ccmpensati 
en is bali pixel cr foil pixel motion ccmpensatico. 

It should be appreciated that the a be re described embodiments maj be in 
pleceotcd io scfuare or hardware. One skilled io the art till apprecia 
tc that the decoder cat be embodied as a semiccndoctcr chip that include 
s logic gates configured tc provide the finctionality d i & c n s s e d a b c r e . 
Fcr example, a hardware description language (DDL), e.g., VERILOG, cao b 
c employed tc synthesize the firmware and the 1 a y c u t cf the logic gates 
for providing tbe necessary functionality described herein tc provide a 
hardware imp 1 emc a t a t i o o of tbe video decode;. 

Figure 18 is a simplified schematic diagram cf a portable electronic de 
vice having decoder circuitry coofigored to utilize hybrid data stractir 
es to minimize memory requirements and tc applj a hybrid f a c t c i i i 3 t i c o/i 
nteger approximation technique to efficiently deccde the bit stream data 

io accordance with coe embodiment cf the invention. Portable electroci 
c device 290 includes central processing unit (CPU) 294, memory 232. dis 
play screen 136 aod decoder ciiciitry 2 98 ? all it cemmnn i ca t i cn fitfe eac 
\ other ever bns 296. Decoder clrcnitry 2 9 S includes logic gates cenfig 
Bred tc provide tbe functionality to reduce memory r eqn i r einent s for the 
video processing and performing inverse mot ico compensation in the ccmpr 
essed domain as described above. It will be apparent tc one skilled in 
the art that deccdei circuity 295 may include memory on a chip ccntaini 
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ng the deccder circuitry or the memory mar be located off-chip. 
Figure 19 is a more detailed schematic diagram of the decoder circuitry 
cf Figure 18 in acccrdaece fith cce embediment cf the indention. I n c ora 
iog bit stream 122 is received by variable length decoder (VLD) circuitr 
y 3 0 0 cf decoder 298. One skilled in the art Fill appreciate thai ileccd 
cr circuitry 298 nay be placed on a semi ccncuc tor chip disposed ce a pri 
oted circuit beard. VLD circuitry 3 0 0 is in c emmun i ca t i en with dcqoanti 
zation circuitry 3G2. VLD circuitry 300 provides motion vector signals 
tc metier] ccmpe d s a t i cn circuitry 3 06 . V t dec processing mestc r y 3 0 8 Here 
S ao iotcioaJ representation cf the videc f i cm deqoantiiaticn circuit rj 
302 that is in the cenpressed domain. DCT circuitry 304 maintains the i 
nterual DCT representation cf the videc f r cm motion compensation circnit 
ry 306. flua length decode (RLD) circuitry 310 aod inverse discrete ccsi 
ue transform C I DCT) circiitry 312 decompress the videc data fcr presents 
t i o n en display screes 136, It s h c o 1 d be appreciated that tbe circuitry 
blocks described herein pjevide the similar f unc t icua 1 i ty tc the blocks 
/stages described with reference tc Figures 2, 6B and 15. 

Id summary, the above described invent ico provides a compressed domain 
video deccder that reduces the arouant cf videc memcrf aod performs inver 
se racticn compensation in tbe compressed domain. .Memory reduction is ac 
hieved by hybrid data structures configured to store and manipulate non- 
zero DCT coefficients cf the reference frame tc define a current frame. 
Tbe hybrid data stricter* icclndes a fixed size array rearing fixed size 
blocks associated with each bled cf a frame cf video data. A variable 
size overflow vector is inclnded in the hybrid data struct ere tc acecmro 
cdate oon-xerc coefficients in excess cf the capacity cf the fixed size 
blccks. The ' annus t cf memory compression achieved thrcugh the ccmpresse 
d domain video decoder is np tc tve times as compared tc a spatial dcir.ai 
n video deccder. Tbe inverse in c t ico compensation for tbe compressed dem 
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a i o video decoder has been optimised tc provide about 15-25 frames per s 
eccnd cf acceptable quality video. A hybrid f actcr i la t ic o/i ntc ge J appro 
xiraation is select iFtly applied tc blocks being decoded. The criteria f 
or determining which i d t e r pc 1 a t i c n cf the f actcii aat ioo/i cteger approxiui 
atico technique tc apply is based c p g d tbe tracsfcim matrix, i.e., facte 
rizatico is applied tc half pixel matrices, while integer approximation 
is applied tc fall pixel matrices. It should be appreciated that tbe cc 
repressed domain pipeline described herein may be incorporated into an MP 
EG-4 simple profile video decoder to cce embed imeit . Fnrtbermore. the e 
mbedimeots enable a variety cf applications tc be pursied, e.g.. pc*er-s 
eatable decoding on battery-operated (CPU c c ns t r a i ne d) devices sod compo 
siting for video conferencing systems. 

With the abcre embediaents in mind, it should be understood that the in 
ve&tion may employ varicis computer-implemented eperatiens iorclviog dat 
a stored in computer systems. These eperatiens include operations reqii 
ring physical manipulation cf physical quantities. Usually, though net 
necessarily, these quantities take the fcrm cf electrical cr magnetic si 
goals capable cf being stored, transferred, combined, c cup a red, and cite 
rffise manipulated. Farther, the ma o i pa I a t i c ds performed arc cfleo refer 
red to io terms, sue fa as producing, identifying, determining, or compart 

The above described invention may be practiced with uthei computer syst 
em cenf i gurat ic as incicding hand-held devices, microprocessor systems, m 
icrcprceessor-based cr programmable consumer electronics, minicomputers. 

mainframe computers and the like. The invention may also be practiced 
in distributing computing environments where tasks are performed by remc 
te processing devices that are linked through a eoroman icat ions network. 

The invention can also be embodied as computer readable cede on a coropu 
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ter readable medium. The computer readable medigro is any data storage d 
evice that can store data irhich can be thereafter read by a computer sys 
tern. Examples of the computer readable medium include hard drives, netw 
(irk attached storage (SAS) . read-only memory, raodem-a ccess niemc r y , CD-R 
OMs, CD-Is, CD-RU r s ? m a g o e 1 1 c tapes, and ether optical and ucn-cptieal da 
ta storage dcT Ices . The computer readable median can also be distribute 
d ever a network coupled computer system sc that the computer readable c 
cde is stored aod executed io a distributed fasbicn. The computer reada 
ble medium may also be an e I ect r cmag net i c carrier wave in which the c crap 
tier cede i s embed i ed . 

Although the fciegcing invest i o o has been described in some detail fcr 
purposes cf clarity cf understanding, it will be apparent that certain c 
hanges and modifications may be practiced within the scope of the append 
ed claims. Accordingly, the present embodiments are to be considered as 

illustrative and net restrictive, and the invent ion is net tc be limite 
d tc the details given herein, but may be modified within the scope and 
einifalents cf the appended claims. Io tbe claims, elements and/cr step 
s dc ect imply any particular order cf operation, unless explicitly star 
ed in the claims. 

4. Brief Description cf the Draftings 

The present indention will be readily understood by the following detai 
led description io conjunction wiih tbe accompanying drawings, and like 
reference numerals designate lile structural elements. 

Tigare 1 is a schematic diagram of a video decoder fcr decoding video d 
ata and performing motion compensation in the spatial domain. 

Figore 2 is a schematic diagram of a video decoder arranged such that i 
nrerse met ice compensation is performed in the compressed domain in accc 



(83) )03-348598 ( P 2 00 3 -*m*& 



rdaoce with one embed i me 1 1 cf the invention. 

Figure 3 is a schematic diagram illustrating inverse met icq c crepe o S 1 1 i c 
o as pefcrmed in the spatial domain. 

Figure 4 is a graph illustrating the peak signal tc ocise ratio CP5NK) 
fcr a pluafity of frames tc demcost rate the effectiveness cf a forced o 
pdate mechanism associated with the H.263 standard. 

Fignre 5 is a schematic diagram i 1 1 est r a t i og the de te rmi nat icn cf half 
pixel value! in the il . 2 63 standard. 

Figure 6A is a schematic diagrams cf a baseline spatial video decoder 

Figure 6B is a schematic diagram cf a compressed domain udec decoder i 
n accordance with one embodiment cf the invention. 

Figure 7 is a bicck diagram illustrating the blcck trans fcrmat iens dori 
ng the ridec encoding and decoding process in accordance with cne embedi 
meet cf the invention. 

Fignre 8 is a schematic diagram illustrating the use cf a separate iodc 
x tc find the starting position cf each S x fi blcck in a raoleogth reprcsc 
n t a t i en . 

Figures 9A and 98 illustrate the sort and merge operations needed tc ad 
d the prediction error tc the prediction fci an array-based data st recto 
re and a list data structure, respectively. 

Fignre 1 fi is a schematic diagram cf a hybrid data stroctnre including a 
n a r r a F Strnctnre and a vector sinctnre tc allow fcr memory compression 

and ccmpotat icoal efficiency in accordance with cne embodiment cf the i 
d v e a t i g n . 

Figures HA tbroigb 1 1 C are graphs ilhstrating the factors evaluated i 
n determining the capacity of the fixed size blocks cf the fixed size ar 
ray and ibe* overflow vector cf the hybrid data strnctrre in accordance w 
ith one embodiment of the invention. 

Figure 12 is a flowchart cf the method operations fcr reducing the memo 
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ry rcqoi rerocnts for decoding a bit stream in accordance with cue cm b c d i m 
en t c I the invention. 

Figure 73 h a schematic diagram illustraticg three examples cf blcck a 
lignroeot to reduce matrix mn I t i p I i ca t i on . 

Figure H is a schematic diagram cf a half pixel i n it r pc I a t i c o fcr a pe 
rfectly a I igotd DCT blcck. 

Figure 15 is a schematic diagram illustrating the rearrangement cf the 
functional blocks cl a compressed domain video deccder to enhance the pr 
ccessing cf the video data in accordance with one embodiment cf the inve 
nt i on. 

Figure 16 is a flowchart diagram ci the method operations fcr perfcrmin 
g inverse met Icq compensation in the compressed domain in accordance wit 
h one embed i me nt cf the invention. 

Figure 17 is a schematic diagram cf the selective application cf the hy- 
brid fader izat i cn/intege r approximation techniqie in accordance with cn 
e embodiment cf the invention. 

Figure 18 is a simplified schematic diagram cf a portable electronic de 
vice having decoder circuitry cenfignred tc utilize hybrid data structur 
«s tc minimize memory requirements and tc apply a hybrid f ac t c r i 2a t i cn/i 
ntegtr approximation technique tc efficiently deccde the bit stream data 

in accordance vith one embodiment cf the invention. 

Figure 19 is a mere detailed schematic diagram cf the deccder circuitry 
of Figure IS in accordance with one embodiment of the invention. 
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